[issue21163] asyncio doesn't warn if a task is destroyed during its execution
Richard Kiss added the comment: I reread more carefully, and I am in agreement now that I better understand what's going on. Thanks for your patience. -- nosy: +Richard.Kiss ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio doesn't warn if a task is destroyed during its execution
Richard Kiss added the comment: The more I use asyncio, the more I am convinced that the correct fix is to keep a strong reference to a pending task (perhaps in a set in the eventloop) until it starts. Without realizing it, I implicitly made this assumption when I began working on my asyncio project (a bitcoin node) in Python 3.3. I think it may be a common assumption for users. Ask around. I can say that it made the transition to Python 3.4 very puzzling. In several cases, I've needed to create a task where the side effects are important but the result is not. Sometimes this task is created in another task which may complete before its child task begins, which means there is no natural place to store a reference to this task. (Goofy workaround: wait for child to finish.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1
New submission from Richard Kiss: import asyncio import os def t1(q): yield from asyncio.sleep(0.5) q.put_nowait((0, 1, 2, 3, 4, 5)) def t2(q): v = yield from q.get() print(v) q = asyncio.Queue() asyncio.get_event_loop().run_until_complete(asyncio.wait([t1(q), t2(q)])) When PYTHONASYNCIODEBUG is set to 1, this causes a strange error: TypeError: send() takes 2 positional arguments but 7 were given See also https://gist.github.com/richardkiss/10564363 -- components: Library (Lib) files: put_get_bug.py messages: 215991 nosy: richard.kiss priority: normal severity: normal status: open title: q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1 type: behavior versions: Python 3.4 Added file: http://bugs.python.org/file34795/put_get_bug.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1
Richard Kiss added the comment: For a reason that I don't understand, this patch to asyncio fixes the problem: --- a/asyncio/tasks.py Mon Mar 31 11:31:16 2014 -0700 +++ b/asyncio/tasks.py Sat Apr 12 20:37:02 2014 -0700 @@ -49,7 +49,8 @@ def __next__(self): return next(self.gen) -def send(self, value): +def send(self, value, *args): return self.gen.send(value) def throw(self, exc): Maybe the problem really is somewhere else, but this works. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21209 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio Task Possibly Incorrectly Garbage Collected
New submission from Richard Kiss: Some tasks created via asyncio are vanishing because there is no reference to their resultant futures. This behaviour does not occur in Python 3.3.3 with asyncio-0.4.1. Also, doing a gc.collect() immediately after creating the tasks seems to fix the problem. Attachment also available at https://gist.github.com/richardkiss/9988156 -- components: Library (Lib) files: asyncio-gc-issue.py hgrepos: 231 messages: 215633 nosy: richard.kiss priority: normal severity: normal status: open title: asyncio Task Possibly Incorrectly Garbage Collected type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file34741/asyncio-gc-issue.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio Task Possibly Incorrectly Garbage Collected
Changes by Richard Kiss h...@richardkiss.com: -- hgrepos: -231 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio task possibly incorrectly garbage collected
Changes by Richard Kiss h...@richardkiss.com: -- title: asyncio Task Possibly Incorrectly Garbage Collected - asyncio task possibly incorrectly garbage collected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio task possibly incorrectly garbage collected
Richard Kiss added the comment: I agree it's confusing and I apologize for that. Background: This multiplexing pattern is used in pycoinnet, a bitcoin client I'm developing at https://github.com/richardkiss/pycoinnet. The BitcoinPeerProtocol class multiplexes protocol messages into multiple asyncio.Queue objects so each interested listener can react. An example listener is in pycoinnet.helpers.standards.install_pong_manager, which looks for ping messages and sends pong responses. When the peer disconnects, the pong manager sees a None (to indicate EOF), and it exits. The return value is uninteresting, so no reference to the Task is kept. My client is in late alpha, and mostly works, but when I tried it on Python 3.4.0, it stopped working and I narrowed it down to this. I'm not certain this behaviour is incorrect, but it's definitely different from 3.3.3, and it seems odd that a GC cycle BEFORE additional references can be made would allow it to work. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio task possibly incorrectly garbage collected
Richard Kiss added the comment: I'll investigate further. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21163] asyncio task possibly incorrectly garbage collected
Richard Kiss added the comment: You were right: adding a strong reference to each Task seems to have solved the original problem in pycoinnet. I see that the reference to the global lists of asyncio.tasks is a weakset, so it's necessary to keep a strong reference myself. This does seem a little surprising. It can make it trickier to create a task that is only important in its side effect. Compare to threaded programming: unreferenced threads are never collected. For example: f = asyncio.Task(some_coroutine()) f.add_callback(some_completion_callback) f = None In theory, the some_coroutine task can be collected, preventing some_completion_callback from ever being called. While technically correct, it does seem surprising. (I couldn't get this to work in a simple example, although I did get it to work in a complicated example.) Some change between 3.3 and 3.4 means garbage collection is much more aggressive at collecting up unreferenced tasks, which means broken code, like mine, that worked in 3.3 fails in 3.4. This may trip up other early adopters of tulip. Maybe adding a do_not_collect=True flag to asyncio.async or asyncio.Task, which would keep a strong reference by throwing it into a singleton set (removing it as a future callback) would bring attention to this subtle issue. Or displaying a warning in debug mode when a Task is garbage-collected before finishing. Thanks for your help. -- resolution: - invalid ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
