[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: Maybe related: bpo-20891 -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue47019] Fatal Python Error in sqlite3 Python 3.10

hydroflask added the comment: So what is causing this bug in 3.10 is the following stack: thread_entry_point() PyGILState_Ensure() PyGILState_Release() ... PyGILState_Ensure() The core issue is that PyGILState_Ensure is getting called while the tstate itself is in the process

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: FYI, PyGILState_Ensure may be called by a thread as many times as you want, and it has been so for a very long time. -- ___ Python tracker

[issue47019] Fatal Python Error in sqlite3 Python 3.10

hydroflask added the comment: If PyGILState_Ensure() has been fixed to become re-entrant during PyGILState_Release() in 3.11+ then I believe that change should be backported to 3.10. @vstinner would know -- ___ Python tracker

[issue47019] Fatal Python Error in sqlite3 Python 3.10

hydroflask added the comment: > All callbacks triggered from external libraries must protect Python C API > calls. You may call it offending, but that is the reality; a callback may > trigger at any point in time, so we need to make sure the GIL is held before > calling any Py API. That is

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: See also Petrs msg400205: > [...] I recommend that any function passed to SQLite (and only those) should > - be named `*_callback`, for clarity > - acquire the GIL at the very start > - release the GIL at the very end --

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: See also bpo-44304 -- nosy: +pablogsal ___ Python tracker ___ ___ Python-bugs-list mailing

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: All callbacks triggered from external libraries must protect Python C API calls. You may call it offending, but that is the reality; a callback may trigger at any point in time, so we need to make sure the GIL is held before calling any Py API. That is

[issue47019] Fatal Python Error in sqlite3 Python 3.10

hydroflask added the comment: I don't see it immediately but I think it's still possible to happen since all the same offending code is in place. There are two reasosn why it probably doesn't happen in 3.11+: 1) because something is deferring calling the finalizer for the zero-ref object

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: > If you connect with check_same_thread=False, I believe the issue may still > present itself on 3.11+ Nope, works swell on my Mac. Do you see this on Linux with latest 3.11? -- ___ Python tracker

[issue47019] Fatal Python Error in sqlite3 Python 3.10

hydroflask added the comment: If you connect with check_same_thread=False, I believe the issue may still present itself on 3.11+ -- ___ Python tracker ___

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: > My recommended fix is to call sqlite3_close() within > Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS, and manually Py_DECREF all > connection-related functions afterward. 3.11 already drops the GIL before sqlite3_close_v2(); backporting this change to

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: > A perfect repro has been attached, should be run on Linux. FTR, the repo works on macOS 12.2.1 as well: $ python3.10 ./sqlite3_fatal_python_error.py Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the

[issue47019] Fatal Python Error in sqlite3 Python 3.10

Erlend E. Aasland added the comment: I'm removing 3.11 from the versions field: $ python3.11 sqlite3_fatal_python_error.py Exception ignored in: Traceback (most recent call last): File "/Users/erlendaasland/src/cpython-build/sqlite3_fatal_python_error.py", line 10, in

[issue47019] Fatal Python Error in sqlite3 Python 3.10

hydroflask added the comment: If you connect to the sqlite3 database in the example without using the factory, it will simply segfault. -- ___ Python tracker ___

[issue47019] Fatal Python Error in sqlite3 Python 3.10

New submission from hydroflask : _destructor in connection.c in Python 3.10+ now calls `PyGILState_Ensure()`, this is a problem because if the destructor is being called while the thread is being torn down it will cause an unbalanced/erroneous call to "PyEval_RestoreThread" in