[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: Closing this since it applies only to our custom tls implementation. Most platforms use native tls now. -- resolution: -> wont fix status: open -> closed ___ Python tracker

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: Making this low priority since it applies only to platforms without Windows and pthread support. -- priority: normal -> low versions: +Python 3.3 -Python 3.2 ___ Python tracker

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: hm, for some reason this patch isn't viewable in side-by-side -- ___ Python tracker ___ ___

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: New patch, based on the cpython tip. -- Added file: http://bugs.python.org/file24959/tlspatch.patch ___ Python tracker ___ _

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: I'll rework this for python 3.x and see where that gets us. -- ___ Python tracker ___ ___ Py

[issue9787] Release the TLS lock during allocations

Martin v. Löwis added the comment: > Note that we are not making any new requirements on python here. But you are. So far, there was no guarantee whatsoever about the state of Python when malloc is called. You are now introducing a requirement that Python must be in a certain state to make it c

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: >The malloc >implementation just has no business looking at the thread state. Of course it does, if it you want to have any hope of instrumenting your python memory usage with detailed python runtime information. Your statement islike saying: "A profi

[issue9787] Release the TLS lock during allocations

Martin v. Löwis added the comment: > You may find this hard to believe, but we do in fact embed python > into other applications. This is actually very easy to believe. > appMalloc, is in this case, the canonical memory allocator in > UnrealEngine. But it could be any other memory allocator s

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: I forgot to add: The API that our (instrumented) malloc implementation is calling is: PyGILState_GetThisThreadState(); -- ___ Python tracker _

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: You may find this hard to believe, but we do in fact embed python into other applications. In this case, it is UnrealEngine, to drive a complex, console based game. Yes, embedding python is much harder than it need be and I'll submit some patches to

[issue9787] Release the TLS lock during allocations

Martin v. Löwis added the comment: What is appMalloc, and what does it have to do with some Python lock? You seem to suggest that some malloc implementations make use of Python interpreter internals. I would call that a bug in the malloc implementation (it violates standard layering assumptio

[issue9787] Release the TLS lock during allocations

Kristján Valur Jónsson added the comment: You're right. Added a new version of the patch. -- Added file: http://bugs.python.org/file18793/tlspatch.patch ___ Python tracker ___ __

[issue9787] Release the TLS lock during allocations

Antoine Pitrou added the comment: You have a bug in PyThread_delete_key_value() (to_free = NULL?). Also, you should move the "/* NB This does *not* free p->value! */" comments at the right places. -- nosy: +pitrou ___ Python tracker

[issue9787] Release the TLS lock during allocations

New submission from Kristján Valur Jónsson : Holding the "keymutex" lock during malloc and free operations is not a good idea. The reason is, that custom implementations of malloc and free, can use the TLS themselves. This is, for example, true in embedded situations, where one wants to repl