[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: Closing this issue since sqlite3_close{,_v2}() is no longer called with the GIL held in 3.11+ -- stage: -> resolved status: open -> closed ___ Python tracker

[issue42698] Deadlock in pysqlite_connection_dealloc()

Erlend E. Aasland added the comment: > Any update on this? I was unable to reproduce this last time we visited this issue, but I'll give it another try. > If there is no immediate plan to make this change you may close the issue :) I do not want to close it until I can a) either prove that

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: Any update on this? I know you wanted to repro but even in the absence of the repro, I think calling sqlite3_close() without releasing the GIL is error-prone. If there is no immediate plan to make this change you may close the issue :) --

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: >> The major problem is that I don't exactly know how to provoke SQLite to >> acquire an internal lock. > IIRC, you can provoke the internal SQLite lock simply by using transaction > control: BEGIN (lock) => COMMIT / ROLLBACK (unlock). Ah okay, so the sequence

[issue42698] Deadlock in pysqlite_connection_dealloc()

Erlend E. Aasland added the comment: > In production the error was happening after 30 minutes of so. Great, thanks. > The major problem is that I don't exactly know how to provoke SQLite to > acquire an internal lock. IIRC, you can provoke the internal SQLite lock simply by using

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: You did the right thing in terms of editing the code. I was not able to get a deadlock with this code which I think should be representative of the error. In production the error was happening after 30 minutes of so. The major problem is that I don't exactly

[issue42698] Deadlock in pysqlite_connection_dealloc()

Erlend E. Aasland added the comment: Thanks! I had to comment out the reg_func() call (it is not defined in test.py, and I don't know what it's supposed to do), and change "with lock.shared_context():" to "with lock:", but I'm unable get a deadlock. I've tried with Python 3.8 though 3.11

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: Unfortunately I am currently unable to write a repro for this bug but I am 100% certain it exists from analyzing the code. In our application (which is a large multithreaded application) after I made the changes consistent with this report the deadlock vanished

[issue42698] Deadlock in pysqlite_connection_dealloc()

Erlend E. Aasland added the comment: hydroflask, can you provide a reproducer? -- nosy: +erlendaasland versions: +Python 3.11 -Python 3.6, Python 3.7, Python 3.8 ___ Python tracker

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: Another comment: if calling sqlite3_close() outside of GIL, then the associated SQL function destructors must take the GIL before calling Py_DECREF -- ___ Python tracker

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: Nevermind it seems that it's legal to call Py_BEGIN_ALLOW_THREADS in tp_dealloc. The fix is then to allow threads around sqlite3_close(). -- ___ Python tracker

[issue42698] Deadlock in pysqlite_connection_dealloc()

hydroflask added the comment: This is also a problem in pysqlite_connection_close() as currently implemented. -- ___ Python tracker ___

[issue42698] Deadlock in pysqlite_connection_dealloc()

Change by hydroflask : -- components: +Extension Modules ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue42698] Deadlock in pysqlite_connection_dealloc()

New submission from hydroflask : pysqlite_connection_dealloc() calls sqlite3_close{,_v2}(). This can cause a deadlock if sqlite3_close() tries to acquire a lock that another thread holds, due to a deadlock between the GIL and an internal sqlite3 lock. This is especially common with