New submission from Glenn Langford:
as_completed() calls _create_and_install_waiters() over all given Futures, even
if some (or all) of the Futures have already completed.
The following fragment of the as_completed code (from 3.3.3):
with _AcquireFutures(fs):
finished = set(
f for f in fs
if f._state in [CANCELLED_AND_NOTIFIED, FINISHED])
pending = set(fs) - finished
waiter = _create_and_install_waiters(fs, _AS_COMPLETED)
installs waiters into Futures contained in fs, rather than pending.
A more efficient approach might be to only install waiters on the pending
subset if necessary. This would be faster and would also result in less dwell
time with all of the Future conditions locked via the _AcquireFutures context
manager.
Also: waiters are appended with the Future condition lock acquired, but are
removed (at the conclusion of as_completed) without the _AcquireFutures
condition lock. Is this correct?
----------
components: Library (Lib)
messages: 208418
nosy: glangford
priority: normal
severity: normal
status: open
title: concurrent.futures.as_completed() installs waiters for already completed
Futures
type: performance
versions: Python 3.3, Python 3.4
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue20297>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
