Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

setting cpu affinity doesn't work on py3 #3973

Closed
Valodim opened this issue Sep 27, 2018 · 9 comments
Closed

setting cpu affinity doesn't work on py3 #3973

Valodim opened this issue Sep 27, 2018 · 9 comments

Comments

@Valodim
Copy link
Contributor

Valodim commented Sep 27, 2018

python[8411]: 2018-09-27 18:54:25,831 - root - 212 - WARNING - None- ***** STARTING SERVER *****
python[8411]: 2018-09-27 18:54:25,848 - root - 215 - WARNING - None- Server /home/valodim/cpy/lib/python3.5/site-packages/synapse/app/homeserver.py v
python[8411]: 2018-09-27 18:54:25,858 - twisted - 243 - WARNING - None- /home/valodim/cpy/lib/python3.5/site-packages/psycopg2/__init__.py:144: built
python[8411]: 2018-09-27 18:54:26,048 - twisted - 243 - ERROR - - ******************************************************
python[8411]: 2018-09-27 18:54:26,049 - twisted - 243 - ERROR - -  Missing package 'affinity' required for cpu_affinity
python[8411]: 2018-09-27 18:54:26,050 - twisted - 243 - ERROR - -  option
python[8411]: 2018-09-27 18:54:26,050 - twisted - 243 - ERROR - -
python[8411]: 2018-09-27 18:54:26,051 - twisted - 243 - ERROR - -  Install by running:
python[8411]: 2018-09-27 18:54:26,051 - twisted - 243 - ERROR - -
python[8411]: 2018-09-27 18:54:26,052 - twisted - 243 - ERROR - -     pip install affinity
python[8411]: 2018-09-27 18:54:26,052 - twisted - 243 - ERROR - -
python[8411]: 2018-09-27 18:54:26,052 - twisted - 243 - ERROR - -
python[8411]: 2018-09-27 18:54:26,053 - twisted - 243 - ERROR - - ******************************************************

the affinity package is installed, I ran pip in that virtualenv to make sure, and checked it's there in site-packages. not sure why synapse can't find the package.

perhaps the setting no longer makes sense with py3? anyways this prevents synapse from starting for me. commenting out the setting works fine, however.

This is on 0.33.5.1

@Half-Shot
Copy link
Collaborator

Probably the first two questions to ask are "did you install this on pip3 rather than pip, which is usually the py2 version" and "how did you install synapse?"

@Valodim
Copy link
Contributor Author

Valodim commented Sep 29, 2018

> bin/pip --version
pip 18.0 from /home/valodim/cpy/lib/python3.5/site-packages/pip (python 3.5)
> bin/pip install affinity
Requirement already satisfied: affinity in ./lib/python3.5/site-packages (0.1.0)
> bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/
[above error]

synapse is installed in the virtualenv with bin/pip install ./synapse

@richvdh richvdh added the py3 label Oct 1, 2018
@hawkowl hawkowl self-assigned this Oct 2, 2018
@hawkowl
Copy link
Contributor

hawkowl commented Oct 2, 2018

This is something that should work on Python 3, so I'll take a look. Thanks for the report, @Valodim .

@logic
Copy link

logic commented Oct 4, 2018

FWIW, it looks like the affinity module hasn't been ported to Python 3, and appears to be a dead project. PyPI says the last release was March 23, 2006, and just importing it in Python3 throws syntax errors around exception syntax:

>>> import affinity
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/srv/matrix.logic.net/venv3.6/lib/python3.6/site-packages/affinity/__init__.py", line 29
    except pywintypes.error, e:
                           ^
SyntaxError: invalid syntax
>>>

@hawkowl
Copy link
Contributor

hawkowl commented Oct 4, 2018

That's rather unfortunate.

However, looking at what it actually wraps, I believe a CFFI-using replacement which covers our purposes (Linux only) would be almost trivial to write, so I'll take a stab at that next week.

@ara4n
Copy link
Member

ara4n commented Oct 4, 2018

Do we even need this on Py3? The reason it was added in the first place was due to https://www.mirantis.com/blog/improve-performance-python-programs-restricting-single-cpu/, which i thought might have been addressed in Python 3.5+?

@hawkowl
Copy link
Contributor

hawkowl commented Oct 4, 2018 via email

@erikjohnston
Copy link
Member

No -- it's up to the operating system to decide where to schedule
something, not the application. Linux schedulers sometimes try and avoid
breaking caches and keep things on the same CPU, but that isn't always the
case.

The problem is that it interacts badly with the implementation of the GIL, since the OS may schedule the threads onto multiple cores, but then the python threads fight/thrash over the GIL locks.

It looks like the GIL may have changed to make it thrash less in python 3.2 http://www.dabeaz.com/python/UnderstandingGIL.pdf ?

If psutil has an implementation, lets just use that. Once we get matrix.org on py3 we should definitely try removing the CPU affinity options to see if the thrashing has indeed been fixed or not, since pinning to a single core does have some performance penalties.

@hawkowl hawkowl moved this from To Do to In Progress: Operational/bug fixes in Superceded by https://github.com/orgs/matrix-org/projects/8 Oct 10, 2018
@hawkowl
Copy link
Contributor

hawkowl commented Oct 12, 2018

Fixed in #4027

@hawkowl hawkowl closed this as completed Oct 12, 2018
Superceded by https://github.com/orgs/matrix-org/projects/8 automation moved this from In Progress: Operational/bug fixes to Done - Operations Oct 12, 2018
@hawkowl hawkowl moved this from Done - Operations to Done - Planned Project in Superceded by https://github.com/orgs/matrix-org/projects/8 Oct 12, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
No open projects
Superceded by https://github.com/orgs...
  
Done - Planned Project
Development

No branches or pull requests

8 participants