[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Andrew Svetlov added the comment: Lukasz, please don't rush. Applied PR makes asyncio more stable, not worse. I see the only way to make it perfect: get rid of weak refs entirely. Otherwise there is always a chance to catch GC run and set element deletion on iteration over the set. Sorry,

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Łukasz Langa added the comment: Note: there's a discussion on GitHub on PR 13971 being a bad solution: https://github.com/python/cpython/pull/13971#issuecomment-500908198 They should be reverted (or fixed forward) for 3.8 and 3.9. I'm OK with this modifying the AbstractEventLoop API if need

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

miss-islington added the comment: New changeset 83abd9658b4845299452be06c9ce92cbceee944d by Miss Islington (bot) in branch '3.8': bpo-36607: Eliminate RuntimeError raised by asyncio.all_tasks() (GH-13971) https://github.com/python/cpython/commit/83abd9658b4845299452be06c9ce92cbceee944d

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Andrew Svetlov : -- resolution: fixed -> stage: resolved -> status: closed -> open ___ Python tracker ___ ___

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

miss-islington added the comment: New changeset 5d1d4e3179ffd4d2d72462d870cf86dcc11450ce by Miss Islington (bot) in branch '3.7': bpo-36607: Eliminate RuntimeError raised by asyncio.all_tasks() (GH-13971) https://github.com/python/cpython/commit/5d1d4e3179ffd4d2d72462d870cf86dcc11450ce

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Andrew Svetlov : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by miss-islington : -- pull_requests: +13843 pull_request: https://github.com/python/cpython/pull/13976 ___ Python tracker ___

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

miss-islington added the comment: New changeset 65aa64fae89a24491aae84ba0329eb8f3c68c389 by Miss Islington (bot) (Andrew Svetlov) in branch 'master': bpo-36607: Eliminate RuntimeError raised by asyncio.all_tasks() (GH-13971)

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by miss-islington : -- pull_requests: +13842 pull_request: https://github.com/python/cpython/pull/13975 ___ Python tracker ___

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Andrew Svetlov : -- versions: +Python 3.9 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Andrew Svetlov : -- keywords: +patch pull_requests: +13838 stage: -> patch review pull_request: https://github.com/python/cpython/pull/13971 ___ Python tracker ___

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Nick Davies added the comment: > Would you prepare a patch for number 3? I will give it a try and see what I come up with. > I am afraid we can add another hard-to-debug multi-threaded problem by > complicating the data structure. Yeah this was my concern too, the adding and removing from

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Andrew Svetlov added the comment: Sorry, I've missed that the loop has hashable requirement already. Would you prepare a patch for number 3? I am afraid we can add another hard-to-debug multi-threaded problem by complicating the data structure. I'm just curious why do you call `all_tasks()`

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Nick Davies added the comment: My preference would actually be number 3 because: 1: I agree that this isn't really a safe option because it could slow things down (possibly a lot) 2: I haven't found this to be rare in my situation but I am not sure how common my setup is. We have a

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Andrew Svetlov added the comment: The fix can be applied to 3.7 and 3.8 only, sorry. Python 3.6 is in security mode now. -- versions: +Python 3.8 -Python 3.6 ___ Python tracker

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Andrew Svetlov added the comment: Thanks for the report! I see 3 ways to fix the bug: 1. Guard _all_tasks with threading.Lock. It hurts performance significantly. 2. Retry list(_all_tasks) call in a loop if RuntimeError was raised. A chance of collision is very low, the strategy is good

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Tim Hatch : -- nosy: +thatch ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Nick Davies : -- type: -> behavior versions: +Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

Change by Sam Dunster : -- nosy: +sdunster ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads

New submission from Nick Davies : This problem was identified in https://bugs.python.org/issue34970 but I think the fix might have been incorrect. The theory in issue34970 was that GC was causing the weakrefset for `all_tasks` to change during iteration. However Weakset provides an