[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: I've now pushed the patch. I hope this won't break anything, closing. -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker

[issue11635] concurrent.futures uses polling

Roundup Robot added the comment: New changeset 4390d6939a56 by Antoine Pitrou in branch '3.2': Issue #11635: Don't use polling in worker threads and processes launched by http://hg.python.org/cpython/rev/4390d6939a56 New changeset a76257a99636 by Antoine Pitrou in branch 'default': Issue #11635

[issue11635] concurrent.futures uses polling

Changes by s7v7nislands : -- nosy: +s7v7nislands ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: After studying the multiprocessing code, it turns out that Queue.get() with a timeout does its own rather high-frequency polling under Windows (see Modules/_multiprocessing/pipe_connection.c). Therefore, here is an updated patch which doesn't have a security

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: Tests now committed, here is a patch without them. -- Added file: http://bugs.python.org/file21373/cfpolling4.patch ___ Python tracker ___

[issue11635] concurrent.futures uses polling

Roundup Robot added the comment: New changeset 76a898433a02 by Antoine Pitrou in branch '3.2': Add tests for the atexit hook in concurrent.futures (part of #11635) http://hg.python.org/cpython/rev/76a898433a02 New changeset d6bbde982c1c by Antoine Pitrou in branch 'default': Add tests for the a

[issue11635] concurrent.futures uses polling

Changes by Antoine Pitrou : Removed file: http://bugs.python.org/file21360/cfpolling3.patch ___ Python tracker ___ ___ Python-bugs-list mailin

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: Oops, test didn't work under Windows. Here is a new patch. -- Added file: http://bugs.python.org/file21361/cfpolling3.patch ___ Python tracker

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: Ok, here is a new patch with an additional test for the atexit hook. If you don't object, I would like to start committing the test changes, and then the code changes themselves. -- stage: needs patch -> patch review versions: +Python 3.2 Added file:

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: > Sorry, I didn't read an error message very carefully. When I apply your patch > I see: > > >>> from concurrent.futures import * > >>> from time import * > >>> t = ThreadPoolExecutor(5) > >>> t.submit(sleep, 100) > > >>> > Error in atexit._run_exitfuncs: >

[issue11635] concurrent.futures uses polling

Brian Quinlan added the comment: Sorry, I didn't read an error message very carefully. When I apply your patch I see: >>> from concurrent.futures import * >>> from time import * >>> t = ThreadPoolExecutor(5) >>> t.submit(sleep, 100) >>> Error in atexit._run_exitfuncs: NameError: global name

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: > Your approach seems workable but your patch allows the interpreter to > exit while work items are still being processed. See the comment at > the top of concurrent/futures/thread.py. Why are you saying that? In my patch, _python_exit() still takes care of joi

[issue11635] concurrent.futures uses polling

Brian Quinlan added the comment: Your approach seems workable but your patch allows the interpreter to exit while work items are still being processed. See the comment at the top of concurrent/futures/thread.py. -- ___ Python tracker

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: > I would suggest that you base your patch on 3.3/default. Well, since the module is new, I think it would be nice to fix such quirks in the bugfix branch as well. So, following the recommended workflow, I've started with a 3.2 patch. -- _

[issue11635] concurrent.futures uses polling

Brian Quinlan added the comment: I would suggest that you base your patch on 3.3/default. -- ___ Python tracker ___ ___ Python-bugs-l

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: (patch is for 3.2, by the way. Perhaps this should only be fixed in default?) -- ___ Python tracker ___ ___

[issue11635] concurrent.futures uses polling

Antoine Pitrou added the comment: Ok, here is a patch. Summary: - leave a minimal amount of polling (every 600 seconds) to avoid blocking forever if there's a bug (shouldn't happen of course, but who knows? especially with multiprocessing) - when wanting to wakeup a worker, put None in its rec

[issue11635] concurrent.futures uses polling

Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.p

[issue11635] concurrent.futures uses polling

New submission from Antoine Pitrou : concurrent.futures uses polling in its worker threads and processes (with a timeout of 0.1). It means that: 1) this prevents CPUs to enter low power states durably 2) it incurs latency when calling shutdown() on an executor (this seems to be the main source o