[issue15751] Support subinterpreters in the GIL state API

Change by Maciej Szulik : -- nosy: +maciej.szulik ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15751] Support subinterpreters in the GIL state API

Change by Paulo Henrique Silva : -- nosy: +phsilva ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15751] Support subinterpreters in the GIL state API

Change by Eric Snow : -- versions: +Python 3.8, Python 3.9 -Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue15751] Support subinterpreters in the GIL state API

Roundup Robot added the comment: New changeset e590c632c9fa by Victor Stinner in branch 'default': Add more checks on the GIL https://hg.python.org/cpython/rev/e590c632c9fa -- ___ Python tracker

[issue15751] Support subinterpreters in the GIL state API

Changes by Eric Snow ericsnowcurren...@gmail.com: -- versions: +Python 3.6 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15751 ___ ___

[issue15751] Support subinterpreters in the GIL state API

Roundup Robot added the comment: New changeset 5d078b0bae75 by Victor Stinner in branch 'default': Issue #19787: PyThread_set_key_value() now always set the value http://hg.python.org/cpython/rev/5d078b0bae75 -- nosy: +python-dev ___ Python tracker

[issue15751] Support subinterpreters in the GIL state API

STINNER Victor added the comment: FYI I fixed a weird behaviour of the PyThread_set_key_value() function. The change has indirectly on impact on test.support.run_in_subinterp(): The impact was a bug. I reverted my changeset and pushed a new changeset which leaves

[issue15751] Support subinterpreters in the GIL state API

STINNER Victor added the comment: FYI I fixed a weird behaviour of the PyThread_set_key_value() function. The change has indirectly on impact on test.support.run_in_subinterp(): http://bugs.python.org/issue19787#msg206015 So my change might have an effect on this issue. --

[issue15751] Support subinterpreters in the GIL state API

STINNER Victor added the comment: They are now two issues (#10915 and this one) with many messages. Both issues are open. Can someone please make a summary? How can we fix the GIL/subinterpreter issue? -- ___ Python tracker rep...@bugs.python.org

[issue15751] Support subinterpreters in the GIL state API

Antoine Pitrou added the comment: I wanted to set issue10915 as duplicate but there is actually a tentative patch there. Unfortunately the discussions are now split apart... -- ___ Python tracker rep...@bugs.python.org

[issue15751] Support subinterpreters in the GIL state API

Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15751 ___ ___ Python-bugs-list

[issue15751] Support subinterpreters in the GIL state API

Changes by Eric Snow ericsnowcurren...@gmail.com: -- nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15751 ___ ___ Python-bugs-list

[issue15751] Support subinterpreters in the GIL state API

Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15751 ___ ___ Python-bugs-list mailing list

[issue15751] Support subinterpreters in the GIL state API

Antoine Pitrou added the comment: Le mercredi 29 août 2012 à 02:19 +, Nick Coghlan a écrit : However, it *doesn't* work (at least, not easily) if the extension itself wants to call back into an interpreter other than the one already associated with the current thread: /* Embedding

[issue15751] Support subinterpreters in the GIL state API

Nick Coghlan added the comment: Yeah, I eventually came around to agreeing that it's better to stick with the Ensure/Release model. SwitchInterpreter gets messy when it comes to managing the lifecycle of temporary thread states. So an EnsureEx/ReleaseEx pair with a new struct that includes

[issue15751] Support subinterpreters in the GIL state API

Graham Dumpleton added the comment: If PyGILState_STATE is a struct, what happens if someone naively does: PyGILState_Release(PyGILState_UNLOCKED) I know they shouldn't, but I actually do this in mod_wsgi in one spot as it is otherwise a pain to carry around the state when I know for sure if

[issue15751] Support subinterpreters in the GIL state API

Mark Hammond added the comment: The GIL state api was mainly interested in the case of a thread which has (possibly) never been seen before calling into Python. IIUC, the proposal here is so that a thread that *has* been seen before can be associated with a thread-state specified by the

[issue15751] Support subinterpreters in the GIL state API

Mark Hammond added the comment: To clarify, I wrote: can be associated with a thread-state specified by the embedding application Where I meant to say: Can be associated with an interpreter state and corresponding thread-state ... Or something like that - it's been a while since I've

[issue15751] Support subinterpreters in the GIL state API

Nick Coghlan added the comment: Thinking about it, I believe there still needs to be a concept of an active thread state TLS key in order to deal with Graham's original problem. Specifically, if PyGILState_EnsureEx is used to associate the thread with a particular interpreter, then subsequent

[issue15751] Support subinterpreters in the GIL state API

Antoine Pitrou added the comment: Le mardi 28 août 2012 à 14:12 +, Nick Coghlan a écrit : old_interp = PyGILState_SwitchInterpreter(target_interp); old_gil = PyGILState_Ensure(); /* Call into Python using target_interp */ PyGILState_Release(old_gil);

[issue15751] Support subinterpreters in the GIL state API

Graham Dumpleton added the comment: Sorry, Mark. Is not for associating thread state specified by embedding application. In simple terms it is exactly like existing PyGILState_Ensure() in that caller doesn't have a thread state, whether it has already been created. Only difference is to allow

[issue15751] Support subinterpreters in the GIL state API

Nick Coghlan added the comment: The reason I'm proposing going back to the original SwitchInterpreter idea is because the EnsureEx idea doesn't nest cleanly - if your thread is already associated with interpreter A, you can't readily call into interpeter B, because the API provides no way to

[issue15751] Support subinterpreters in the GIL state API

Graham Dumpleton added the comment: Nick. Valid point. I guess I hadn't been thinking about case of one thread calling out of one interpreter and then into another, as I don't do it in mod_wsgi and even I regard doing that as partly evil. Does that mean this switch interpreter call somehow

[issue15751] Support subinterpreters in the GIL state API

Nick Coghlan added the comment: No, those wouldn't change at all. However, after thinking about it further, I'm leaning back towards the option of an EnsureEx/ReleaseEx pair that allows the previous interpreter to be reported and restored along with the GIL state. I'd also like to make the

[issue15751] Support subinterpreters in the GIL state API

Graham Dumpleton added the comment: So you are saying that as user of this I am meant to call it as: PyGILState_INFO info; PyGILState_EnsureEx(interp, info); ... PyGILState_ReleaseEx(info); What is the potential error code from PyGILState_EnsureEx() considering that right now for

[issue15751] Support subinterpreters in the GIL state API

Nick Coghlan added the comment: Currently, both PyGILState_Ensure and PyGILState_Release just call PyFatal_Error if anything goes wrong. With the *Ex versions, it would be possible to replace those fatal errors with more conventional error handling. For Ensure, the likely error is failing to

[issue15751] Support subinterpreters in the GIL state API

Nick Coghlan added the comment: And the current autoTLSkey could move into the interpreter state object? I like it - that's a lot more flexible than the approach I was thinking of. -- title: Add PyGILState_SwitchInterpreter - Support subinterpreters in the GIL state API

[issue15751] Support subinterpreters in the GIL state API

Graham Dumpleton added the comment: It is past my bed time and not thinking straight, but I believe Antoine is aligned with what I had in mind, as need multiple thread states per OS thread where each is associated with separate interpreter. My main reason for allowing NULL to EnsureEX rather