[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset a36adfa6bbf5e612a4d4639124502135690899b8 by Victor Stinner in branch 'master': bpo-39877: 4th take_gil() fix for daemon threads (GH-19080) https://github.com/python/cpython/commit/a36adfa6bbf5e612a4d4639124502135690899b8 --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18437 pull_request: https://github.com/python/cpython/pull/19080 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 29356e03d4f8800b04f799efe7a10e3ce8b16f61 by Victor Stinner in branch 'master': bpo-39877: Fix take_gil() for daemon threads (GH-19054) https://github.com/python/cpython/commit/29356e03d4f8800b04f799efe7a10e3ce8b16f61 --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18405 pull_request: https://github.com/python/cpython/pull/19054 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: The initial issue is now fixed. I close the issue. take_gil() only checks if the thread must exit once the GIL is acquired. Maybe it would be able to exit earlier, but I took the safe approach. If we must exit, drop the GIL and then exit. That's basically

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: I created bpo-39941 "multiprocessing: Process.join() should emit a warning if the process is killed by a signal" which may help to detect the issue earlier on any platform, not only on FreeBSD. -- ___ Python

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset b4698ecfdb526e0a9f5fa6ef0f8e1d8cca500203 by Victor Stinner in branch 'master': bpo-39877: Deprecate PyEval_InitThreads() (GH-18892) https://github.com/python/cpython/commit/b4698ecfdb526e0a9f5fa6ef0f8e1d8cca500203 --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18249 pull_request: https://github.com/python/cpython/pull/18892 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 175a704abfcb3400aaeb66d4f098d92ca7e30892 by Victor Stinner in branch 'master': bpo-39877: PyGILState_Ensure() don't call PyEval_InitThreads() (GH-18891) https://github.com/python/cpython/commit/175a704abfcb3400aaeb66d4f098d92ca7e30892

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: I tested (run multiple times) daemon_threads_exit.py with slow_exit.patch: no crash. I also tested (run multiple times) stress.py + sleep_at_exit.patch of bpo-37135: no crash. And I tested asyncio_gc.py of bpo-19466: no crash neither. Python finalization

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: > The problem is that Python already freed the memory of all PyThreadState > structures, whereas PyEval_RestoreThread(tstate) dereferences tstate to get > the _PyRuntimeState structure: Funny/not funny, bpo-36818 added a similar bug with commit

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18248 pull_request: https://github.com/python/cpython/pull/18891 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 9229105f19705f72e553cf066751ac47c7b7 by Victor Stinner in branch 'master': bpo-39877: take_gil() checks tstate_must_exit() twice (GH-18890) https://github.com/python/cpython/commit/9229105f19705f72e553cf066751ac47c7b7 --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18247 pull_request: https://github.com/python/cpython/pull/18890 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18245 pull_request: https://github.com/python/cpython/pull/1 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 85f5a69ae1541271286bb0f0e0303aabf792dd5c by Victor Stinner in branch 'master': bpo-39877: Refactor take_gil() function (GH-18885) https://github.com/python/cpython/commit/85f5a69ae1541271286bb0f0e0303aabf792dd5c --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18242 pull_request: https://github.com/python/cpython/pull/18885 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 111e4ee52a1739e7c7221adde2fc364ef4954af2 by Victor Stinner in branch 'master': bpo-39877: Py_Initialize() pass tstate to PyEval_InitThreads() (GH-18884) https://github.com/python/cpython/commit/111e4ee52a1739e7c7221adde2fc364ef4954af2

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18241 pull_request: https://github.com/python/cpython/pull/18884 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 3225b9f9739cd4bcca372d0fa939cea1ae5c6402 by Victor Stinner in branch 'master': bpo-39877: Remove useless PyEval_InitThreads() calls (GH-18883) https://github.com/python/cpython/commit/3225b9f9739cd4bcca372d0fa939cea1ae5c6402 --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18240 pull_request: https://github.com/python/cpython/pull/18883 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18211 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/18854 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: I reopen the issue, my change introduced a *new* issue. While trying to fix bpo-19466, work on PR 18848, I noticed that my commit eb4e2ae2b8486e8ee4249218b95d94a9f0cc513e introduced a race condition :-( The problem is that while the main thread is executing

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18210 pull_request: https://github.com/python/cpython/pull/18848 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: Ok, it should now be fixed. Let me try to revisit the old bpo-19466 issue. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset eb4e2ae2b8486e8ee4249218b95d94a9f0cc513e by Victor Stinner in branch 'master': bpo-39877: Fix PyEval_RestoreThread() for daemon threads (GH-18811) https://github.com/python/cpython/commit/eb4e2ae2b8486e8ee4249218b95d94a9f0cc513e --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: New changeset 7b3c252dc7f44d4bdc4c7c82d225ebd09c78f520 by Victor Stinner in branch 'master': bpo-39877: _PyRuntimeState.finalizing becomes atomic (GH-18816) https://github.com/python/cpython/commit/7b3c252dc7f44d4bdc4c7c82d225ebd09c78f520 --

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18174 pull_request: https://github.com/python/cpython/pull/18816 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18169 pull_request: https://github.com/python/cpython/pull/18811 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : -- pull_requests: +18166 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18808 ___ Python tracker ___

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: Patch to get a cleaner error if the bug occurs, but also to make the bug more reliable: diff --git a/Python/ceval.c b/Python/ceval.c index ef4aac2f9a..8bf1e4766d 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -382,7 +382,8 @@ PyEval_SaveThread(void)

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: > This issue should be the root cause of bpo-39088: "test_concurrent_futures > crashed with python.core core dump on AMD64 FreeBSD Shared 3.x", (...) I marked bpo-39088 as a duplicate of this issue. -- ___ Python

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

STINNER Victor added the comment: To reproduce the bug on Linux: * apply attached slow_exit.patch to sleep 1 second just before exiting the process, in the main() function * run: ./python daemon_threads_exit.py # attached file Example: --- $ ./python daemon_threads_exit.py Start 100 daemon

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

Change by STINNER Victor : Added file: https://bugs.python.org/file48958/daemon_threads_exit.py ___ Python tracker ___ ___ Python-bugs-list

[issue39877] Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process

New submission from STINNER Victor : Sometimes, test_multiprocessing_spawn does crash in PyEval_RestoreThread() on FreeBSD with a coredump. This issue should be the root cause of bpo-39088: "test_concurrent_futures crashed with python.core core dump on AMD64 FreeBSD Shared 3.x", where the