[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Roundup Robot added the comment: New changeset b52113fb58a5 by Guido van Rossum in branch '3.4': asyncio: Change as_completed() to use a Queue, to avoid O(N**2) behavior. Fixes issue #20566. http://hg.python.org/cpython/rev/b52113fb58a5 -- ___ Python

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Changes by Guido van Rossum : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Roundup Robot added the comment: New changeset 6e04027ed53e by Guido van Rossum in branch 'default': asyncio: Change as_completed() to use a Queue, to avoid O(N**2) behavior. Fixes issue #20566. http://hg.python.org/cpython/rev/6e04027ed53e -- nosy: +python-dev

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Guido van Rossum added the comment: Note: The new version does *not* change the semantics as mentioned in msg210709. Nobody should depend on those semantics anyway. -- ___ Python tracker __

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Changes by Guido van Rossum : -- Removed message: http://bugs.python.org/msg210715 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Changes by Guido van Rossum : -- Removed message: http://bugs.python.org/msg210765 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Guido van Rossum added the comment: Everyone interested, I plan to push the latest version on view in Rietveld tomorrow: http://codereview.appspot.com/61210043 It's not as drastic a rewrite as my original attempt; Glenn's idea of using a Queue worked out great! -- ___

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Glenn Langford added the comment: > OK, code is ready for review at > http://code.google.com/p/tulip/source/detail?r=674355412f33 This looks like a link to an old revision. -- ___ Python tracker _

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Guido van Rossum added the comment: OK, code is ready for review at http://code.google.com/p/tulip/source/detail?r=674355412f33 -- ___ Python tracker ___ ___

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Guido van Rossum added the comment: I'm looking into a solution for this. The idea is pretty straightforward: http://codereview.appspot.com/61210043. This needs more code to support the optional timeout feature, and it now returns Futures instead of coroutines (which I think is fine). But to

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Glenn Langford added the comment: > Glenn, what led you to discover this? It was found by code inspection. I recently started looking at concurrent.futures, really just curious as to how futures were implemented. Because one of the concurrent.futures bugs I raised also applied to asyncio, I s

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Guido van Rossum added the comment: BTW, just curious: Glenn, what led you to discover this? -- ___ Python tracker ___ ___ Python-bugs

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Changes by Yury Selivanov : -- nosy: +yselivanov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Guido van Rossum added the comment: Yup, I remember feeling a bit guilty doing it this way, but at least the semantics are correctly, and there were bigger fish to fry. Thanks for the test code that proves the issue! I assume it can be fixed by not using _wait() but some custom approach. If we

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Changes by Glenn Langford : Added file: http://bugs.python.org/file34001/output.txt ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

Changes by Glenn Langford : Added file: http://bugs.python.org/file34000/test_thrash.py ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue20566] asyncio as_completed() thrashes adding and removing callbacks

New submission from Glenn Langford: In asyncio, tasks.py as_completed() appears to trigger adding and removing callbacks multiple times for the pending set of futures, each time a single future completes. For example, to wait for 5 futures which complete at different times: - as_completed() u