[issue17263] crash when tp_dealloc allows other threads

Change by Charles-François Natali : -- nosy: -neologix ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue17263] crash when tp_dealloc allows other threads

Irit Katriel added the comment: Is this a python 2-only issue? -- nosy: +iritkatriel ___ Python tracker ___ ___ Python-bugs-list

[issue17263] crash when tp_dealloc allows other threads

Changes by Chris Kaynor ckay...@zindagigames.com: -- nosy: +DragonFireCK ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: And here's a patch. -- keywords: +patch Added file: http://bugs.python.org/file29244/thread_local_concurrent.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263

[issue17263] crash when tp_dealloc allows other threads

Antoine Pitrou added the comment: And here's a patch. Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK macros from pystate.c? That said, I doubt this is the issue here. We are removing a string key pointing to a localdummy object. Both are small atomic types not handled

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Btw., where we are at this issue - I have seen many more loops over the threads (via PyThreadState_Next). I have a bad feeling that many of these loops have similar issues. In this case, I am also not sure anymore that it really was a problem. I originally

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK macros from pystate.c? The macro-names HEAD_LOCK/HEAD_UNLOCK irritates me a bit. Protecting only the head would not be enough. Any tstate object could be invalidated. But actually, it

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK macros from pystate.c? I don't like holding locks before calling alien code, it's a recipe for deadlocks: for example, if another thread-local object was deallocated as part of

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK macros from pystate.c? I don't like holding locks before calling alien code, it's a recipe for deadlocks: for example, if another thread-local object was deallocated as part of the

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Btw., this turns out to be at least 4 kind of separate bugs: 1. The crash from the testcase - when the interpreter shuts down. 2. Maybe the crash from my musicplayer app - if that is a different one. But very related to the first one. 3. Many loops over the

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: The symbols are there because it is a library which exports all the symbols. Other debugging information are not there and I don't know any place where I can get them. It currently cannot work on Linux in the same way because the GUI is Cocoa only right now.

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: Note that in threadmodule.c, in local_clear, we are iterating through all threads: In PyDict_DelItem, if the GIL is released and meanwhile, the list of threadstates is altered, is that a problem for this loop? So maybe tstate becomes invalid

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: I don't know how OS X crash report works, but it seems to have at least some debug info available, since some ymbols are resolved in the backtrace. You might be able to get more info with gdb, with something like: gdb /path/to/python (gdb) info line

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: Alright, here's what's going on. When the main thread exits, it triggers the interpreter shutdown, which clears all the tstates in PyInterpreterState_Clear(): void PyInterpreterState_Clear(PyInterpreterState *interp) { PyThreadState *p;

[issue17263] crash when tp_dealloc allows other threads

Antoine Pitrou added the comment: Albert, this happens because daemon threads continue running during interpreter shutdown. I suppose the problem goes away if you make the thread non-daemonic? This shouldn't be a problem in Python 3 where Python threads cannot switch during shutdown.

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: This shouldn't be a problem in Python 3 where Python threads cannot switch during shutdown. What happens if the GIL is relased during shutdown? Also, I'm a bit worried about this code: void PyThreadState_Clear(PyThreadState *tstate) { if

[issue17263] crash when tp_dealloc allows other threads

Antoine Pitrou added the comment: What happens if the GIL is relased during shutdown? In PyEval_RestoreThread(), any thread other than the main thread trying to take the GIL will immediately exit: take_gil(tstate); if (_Py_Finalizing tstate != _Py_Finalizing) {

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Note that in my original application where I encountered this (with sqlite), the backtrace looks slightly different. It is at shutdown, but not at interpreter shutdown - the main thread is still running. https://github.com/albertz/music-player/issues/23 I was

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: Note that in my original application where I encountered this (with sqlite), the backtrace looks slightly different. It is at shutdown, but not at interpreter shutdown - the main thread is still running. Could you post a traceback of this crash?

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Here is one. Others are in the issue report on GitHub. In Thread 5, the PyObject_SetAttr is where some attribute containing a threading.local object is set to None. This threading.local object had a reference to a sqlite connection object (in some TLS

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: Here is one. Others are in the issue report on GitHub. Yes, I've seen it, but I'd need a backtrace with line numbers (like the one you posted above). thread 5 is crashing, but I don't know where. -- ___

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: Sadly, that is quite complicated or almost impossible. It needs the MacOSX system Python and that one lacks debugging information. I just tried with the CPython vom hg-2.7. But it seems the official Python doesn't have objc bindings (and I also need Cocoa

[issue17263] crash when tp_dealloc allows other threads

Antoine Pitrou added the comment: I have two questions: - how do you know the crash really happens because of thread 5? - when the thread.local object is being deleted, has another thread just started looking up its attributes? -- ___ Python tracker

[issue17263] crash when tp_dealloc allows other threads

Antoine Pitrou added the comment: Another question: are threads being started or stopped while the thread local object is being deleted? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: - how do you know the crash really happens because of thread 5? All other threads are blocked on locks or condition variables, it's the only runnable thread. Another question: are threads being started or stopped while the thread local object is

[issue17263] crash when tp_dealloc allows other threads

Antoine Pitrou added the comment: - how do you know the crash really happens because of thread 5? All other threads are blocked on locks or condition variables, it's the only runnable thread. Hm, you are right. Another question: are threads being started or stopped while the thread

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: The latest 2.7 hg still crashes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list

[issue17263] crash when tp_dealloc allows other threads

Albert Zeyer added the comment: The backtrace: Thread 0:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib 0x7fff8a54e386 __semwait_signal + 10 1 libsystem_c.dylib 0x7fff85e30800 nanosleep + 163 2 libsystem_c.dylib

[issue17263] crash when tp_dealloc allows other threads

Charles-François Natali added the comment: Could you try with recent checkout of python 2.7? I wonder if this could be an occurrence of issue #13992 fixed by Antoine a couple months ago. -- nosy: +neologix ___ Python tracker rep...@bugs.python.org

[issue17263] crash when tp_dealloc allows other threads

New submission from Albert Zeyer: If you have some Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS in some tp_dealloc and you use such objects in thread local storage, you might get crashes, depending on which thread at what time is trying to cleanup such object. I haven't fully figured out the

[issue17263] crash when tp_dealloc allows other threads

Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___