[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: I managed to reproduce the bug.py crash with main branch up to commit 12c0012cf97d21bc637056983ede0eaf4c0d9c33 . -- Added file: https://bugs.python.org/file50700/bug.py_asyncio_cpustressed-crash.log ___ Python

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: bisect of main for bug.py with each local python builds get me to commit b127e70a8a682fe869c22ce04c379bd85a00db67 "bpo-46070: Fix asyncio initialization guard (GH-30423)" as the one that fixed bug.py most of the time. At times I can make bug.py segfault be

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: While bisecting main branch I did not only get segfaults but also exceptions, namely: $ ./python ../python-crash-kodi/sqlite3_crash.py Exception ignored deletion of interned string failed: Traceback (most recent call last): File

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: 3.10 branch is fixed if I revert e6bb17fe29713368e1fd93d9ac9611017c4f570c bpo-46070: _PyGC_Fini() untracks objects (GH-30577). Be it if I revert it atop current head 9006b4471cc4d74d0cbf782d841a330b0f46a3d0 or if I test the commit before

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: I did 3.9 branch bisect and commit 52937c26adc35350ca0402070160cf6dc838f359 bpo-46070: _PyGC_Fini() untracks objects (GH-30577) (GH-30580) is the one that broke 3.9 . While with my main branch builds this commit is fine, it is not for the 3.9 branches.

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: By "It is fixed in main branch commit 12c0012cf97d21bc637056983ede0eaf4c0d9c33 ." I mean that this commit is good not that this commit fixes the issue. -- ___ Python tracker

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Alban Browaeys added the comment: sqlite3_crash.py does not crashes on python 3.9 below and equal 3.9.9 and python main branch 12c0012cf97d21bc637056983ede0eaf4c0d9c33. I confirm it crashes on python 3.9.10, 3.9.11, 3.10 branch commit 9006b4471cc4d74d0cbf782d841a330b0f46a3d0 . It is fixed

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- Removed message: https://bugs.python.org/msg411050 ___ Python tracker ___ ___ Python-bugs-list

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: Another measure using the command: PYTHONHASHSEED=0 ./python -X showrefcount -c pass I had to run the command 20 times to get a stable value, I don't know why. main branch: [21981 refs, 5716 blocks] PR: [21887 refs, 5667 blocks] => the PR removes 94

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by jokot3 : -- nosy: +jokot3 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: I created bpo-46368: "faulthandler: add the ability to dump all interpreters, not only the current interpreter". -- ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: pyobject_ob_interp.patch: Quick & dirty patch that I wrote to add PyObject.ob_interp, store in which interpreter an object has been created. -- Added file: https://bugs.python.org/file50560/pyobject_ob_interp.patch

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: Victor: > (*) I made the GC state per-interpreter: commit > 7247407c35330f3f6292f1d40606b7ba6afd5700 (Nov 20, 2019) Eric Snow: > FYI, this was done by me in an earlier comment which we ended up reverting. Later you basically un.reverted that. Well, I

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: It would be nice to add some tests. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: New changeset 52937c26adc35350ca0402070160cf6dc838f359 by Victor Stinner in branch '3.9': bpo-46070: _PyGC_Fini() untracks objects (GH-30577) (GH-30580) https://github.com/python/cpython/commit/52937c26adc35350ca0402070160cf6dc838f359 --

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

miss-islington added the comment: New changeset e6bb17fe29713368e1fd93d9ac9611017c4f570c by Miss Islington (bot) in branch '3.10': bpo-46070: _PyGC_Fini() untracks objects (GH-30577) https://github.com/python/cpython/commit/e6bb17fe29713368e1fd93d9ac9611017c4f570c --

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: I tested manually my fix GH-30580 using: * (1) attached win_py399_crash_reproducer.py * (2) https://bugs.python.org/issue46070#msg410447 mthod Without my fix, I can easily reproduce the crash with (1) and (2). With my fix, I can no longer reproduce the

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- pull_requests: +28779 pull_request: https://github.com/python/cpython/pull/30580 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by miss-islington : -- pull_requests: +28778 pull_request: https://github.com/python/cpython/pull/30579 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by miss-islington : -- pull_requests: +28777 pull_request: https://github.com/python/cpython/pull/30578 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: New changeset 1a4d1c1c9b08e75e88aeac90901920938f649832 by Victor Stinner in branch 'main': bpo-46070: _PyGC_Fini() untracks objects (GH-30577) https://github.com/python/cpython/commit/1a4d1c1c9b08e75e88aeac90901920938f649832 --

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Eric Snow added the comment: > (*) I made the GC state per-interpreter: commit > 7247407c35330f3f6292f1d40606b7ba6afd5700 (Nov 20, 2019) FYI, this was done by me in an earlier comment which we ended up reverting. Later you basically un.reverted that. > The bug is that a C function object

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: This issue has a complex history. (*) I made the GC state per-interpreter: commit 7247407c35330f3f6292f1d40606b7ba6afd5700 (Nov 20, 2019) (*) This change triggered a _PyImport_FixupExtensionObject() bug in sub-interpreter, I fixed it with commit

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: Oh. I managed to write a simple fix which doesn't require to revert the whole "per-interpreter GC" change: GH-30577. -- ___ Python tracker

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- pull_requests: +28776 pull_request: https://github.com/python/cpython/pull/30577 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: When the crash occurs, the _sre.compile function is not destroyed in the interpreter which created the function. The crash is related to _sre.compile method. This method is created in PyInit__sre() called by "import _sre". On Windows, the _sre module is

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: I wrote 3 scripts to reproduce the bug in a more reliable way. So I just have to type "bisect" and it runs the test 12 times. (1) bisect.bat: --- @"C:\vstinner\python\3.9\PCbuild\amd64\python_d.exe" bisect.py --- (2) bisect.py: --- import subprocess import

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: I modified PR 30565 (3.10) and PR 30566 (3.9) to fix the ABI. I added _PyGC_GetState() which always use PyInterpreterState.gc of the main interpreter. -- ___ Python tracker

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Eric Snow added the comment: > adding a new "gc" member in the _PyRuntimeState structure also causes the ABI > CI check to fail. What if you move it to the end of the struct? -- ___ Python tracker

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: I prepared 3 pull requests to revert the commit 7247407c35330f3f6292f1d40606b7ba6afd5700: * PR 30564: main branch * PR 30565: 3.10 branch * PR 30566: 3.9 branch The problem is that the "Check if the ABI has changed" CI job fails in 3.9 and 3.10 branches.

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- pull_requests: +28769 pull_request: https://github.com/python/cpython/pull/30566 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- pull_requests: +28768 pull_request: https://github.com/python/cpython/pull/30565 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- pull_requests: +28767 pull_request: https://github.com/python/cpython/pull/30564 ___ Python tracker ___

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

STINNER Victor added the comment: This bug is hard to reproduce for different reasons: * It occurs randomly: I need between 1 and 50 attempts to reproduce the bug using win_py399_crash_reproducer.py * So far, the bug was only reproduced on Windows. * I failed to reproduce the crash on

[issue46070] [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)

Change by STINNER Victor : -- title: [subinterpreters] asyncio crash when importing _asyncio in subinterpreter (Python 3.8 regression) -> [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression) ___ Python