[issue23209] asyncio: break some cycles
STINNER Victor added the comment: I noticed that _ProactorBasePipeTransport doesn't clear its reference to the socket when it is closed. I changed this in the following commit (now merged in Python 3.4 3.5): https://code.google.com/p/tulip/source/detail?r=61ce7def97272ab1a6488545dc392160c2fbe316 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
STINNER Victor added the comment: I closed the issue #23225, so I'm also closing this issue. Thanks again Martin. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
Roundup Robot added the comment: New changeset 1544bdc409be by Victor Stinner in branch '3.4': Issue #23209, #23225: selectors.BaseSelector.close() now clears its internal https://hg.python.org/cpython/rev/1544bdc409be New changeset 6e7403bc906f by Victor Stinner in branch 'default': Issue #23209, #23225: selectors.BaseSelector.get_key() now raises a https://hg.python.org/cpython/rev/6e7403bc906f -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
Martin Richard added the comment: I updated the selector patch so BaseSelector.get_key() raises KeyError if the mapping is None. All the (non skipped) tests in test_selectors.py passed. Anyway, if there is an other problem with freeing the mapping object (I don't know, maybe reopening a loop may be considered?) this patch can probably be dropped. Since this cycle is broken when the loop is closed, the objects will likely be collected once the program terminates. -- Added file: http://bugs.python.org/file37681/break-selector-map-cycle.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
STINNER Victor added the comment: I opened the issue #23225 selectors: raise an exception if the selector is closed which is a different approach (but it should also fix the reference cycle, I kept the self._map = None change). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
New submission from Martin Richard: Hi, I would like to submit 3 trivial modifications which break a cycle each. It is not much, but those three cycles caused a lot of objects to be garbage collected. They can now be freed using the reference counting mechanism, and therefore, reduce the latency that may be involved by the work of the garbage collector in a long living process. In asyncio/base_subprocess.py: WriteSubprocessPipeProto.proc is a reference to a BaseSubprocessTransport object, which holds a reference to the WriteSubprocessPipeProto in self._protocol. I break the cycle in the protocol at the end of connection_lost(). In asyncio/futures.py: wrap_future() defines a lambda which uses a variable defined in the function, therefore creating a closure, referencing the wrap_future() function and creating a cycle. In the (really trivial) patch, the lambda uses the argument future instead of the fut variable defined in a closure. The closure is not needed anymore. This single cycle is very common, because caused when one uses getaddrinfo(). In asyncio/selectors.py: _BaseSelectorImpl._map keeps a reference to the _SelectorMapping object, which also references the selector with _SelectorMapping._selector. The reference to the map in the selector is cleared once the selector is closed. -- files: break-some-cycles.diff keywords: patch messages: 233770 nosy: martius priority: normal severity: normal status: open title: asyncio: break some cycles Added file: http://bugs.python.org/file37657/break-some-cycles.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
Changes by Martin Richard mart...@martiusweb.net: -- components: +asyncio nosy: +gvanrossum, haypo, yselivanov type: - performance versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
Guido van Rossum added the comment: All three changes look good to me. The selectors.py fix should be applied to CPython first; the others to Tulip first. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
Roundup Robot added the comment: New changeset 376c5398f28d by Victor Stinner in branch '3.4': Issue #23209: Break some reference cycles in asyncio. Patch written by Martin https://hg.python.org/cpython/rev/376c5398f28d -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
STINNER Victor added the comment: Hi Martin, thanks for the patch. It looks good to me. I applied it to Tulip, Python 3.4 and Python 3.5. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
Roundup Robot added the comment: New changeset 7438f2e30908 by Victor Stinner in branch '3.4': Issue #23209: Revert change on selectors, test_selectors failed. https://hg.python.org/cpython/rev/7438f2e30908 New changeset 27cbc877447b by Victor Stinner in branch 'default': (Merge 3.4) Issue #23209: Revert change on selectors, test_selectors failed. https://hg.python.org/cpython/rev/27cbc877447b -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23209] asyncio: break some cycles
STINNER Victor added the comment: Ooops, test_selectors fails because get_key() raises TypeError: 'NoneType' object is not subscriptable when the selector is closed. A different fix should be written. I'm now using tox to run the Tulip test suite, I'm surprised that I didn't notice the failure. It looks like test_selectors is not executed. runtests.py searchs for test classes with a name ending with Tests. I will fix that. -- resolution: fixed - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
