[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 6846d6712a0894f8e1a91716c11dd79f42864216 by Miss Islington (bot) 
in branch '3.10':
bpo-45262, asyncio: Fix cache of the running loop holder (GH-28796) (GH-28816)
https://github.com/python/cpython/commit/6846d6712a0894f8e1a91716c11dd79f42864216


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[STINNER Victor]

STINNER Victor  added the comment:

Is Matthias Reichl's fix enough to use asyncio in a subinterpreter?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[miss-islington]

miss-islington  added the comment:


New changeset 87f0156a229e4cda92ad8e50645c5a71030caf7c by Miss Islington (bot) 
in branch '3.9':
bpo-45262, asyncio: Fix cache of the running loop holder (GH-28796)
https://github.com/python/cpython/commit/87f0156a229e4cda92ad8e50645c5a71030caf7c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[miss-islington]

Change by miss-islington :


--
pull_requests: +27134
pull_request: https://github.com/python/cpython/pull/28816

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +27133
pull_request: https://github.com/python/cpython/pull/28815

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 392a89835371baa0fc4bf79ae479abb80661f57d by Matthias Reichl in 
branch 'main':
 bpo-45262, asyncio: Fix cache of the running loop holder (GH-28796)
https://github.com/python/cpython/commit/392a89835371baa0fc4bf79ae479abb80661f57d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[Matthias Reichl]

Matthias Reichl  added the comment:

We were hitting the same issue in kodi, which uses embedded sub-interpreters to 
run python addons, after one of the addons was switched to asyncio.

The immediate cause of the assertion failure is a use-after-free issue from the 
running loop holder cache:

When the running loop holder is deallocated (which happens eg on interpreter 
shutdown) cached_running_holder holds a dangling pointer.

A subsequent call to get_running_loop() may then pick that up and boom.

While probably not a full fix for this issue I think it would be good to fix 
the use-after-free problem - there could be other code paths that lead to this 
situation. I've created a github pull request for that

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[Matthias Reichl]

Change by Matthias Reichl :


--
keywords: +patch
nosy: +HiassofT
nosy_count: 4.0 -> 5.0
pull_requests: +27124
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28796

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[STINNER Victor]

STINNER Victor  added the comment:

> "Assertion failed: Py_IS_TYPE(rl, _Type), file 
> D:\a\1\s\Modules_asynciomodule.c, line 261"

The _asyncio extension uses static types and doesn't implement the multiphase 
initialization API. It doesn't work well with your "deinitialize/initialize the 
interpreter" use case.

The _asyncio extension should use heap types and the multiphase init API.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I guess the fix requires switching C Extension types from static to heap for 
_asyncio module.
It is possible for sure but requires a non-trivial amount of work.

We need a champion for the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[Benjamin Schiller]

New submission from Benjamin Schiller :

We have embedded Python in our application and we deinitialize/initialize the 
interpreter at some point of time. If a simple script with a thread that sleeps 
with asyncio.sleep is loaded before and after the re-initialization, then we 
get the following assertion in the second run of the python module:

"Assertion failed: Py_IS_TYPE(rl, _Type), file 
D:\a\1\s\Modules_asynciomodule.c, line 261"

Example to reproduce this crash: 
https://github.com/benjamin-sch/asyncio_crash_in_second_run

--
components: asyncio
messages: 402412
nosy: asvetlov, benjamin-sch, yselivanov
priority: normal
severity: normal
status: open
title: crash if asyncio is used before and after re-initialization if using 
python embedded in an application
type: crash
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com