Nick Coghlan added the comment:
The reason I'm proposing going back to the original SwitchInterpreter idea is
because the EnsureEx idea doesn't nest cleanly - if your thread is already
associated with "interpreter A", you can't readily call into "interpeter B",
because the API provides no way to correctly restore the associated interpreter
back to interpreter A when you're done.
EnsureEx works fine if extension modules are not aware of multiple interpreters:
/* Embedding application (GIL always unlocked) */
gilstate = PyGILState_EnsureEx(interp_A);
/* Python code and extension code happens */
/* Callback needs to reach back into Python */
cb_gilstate = PyGILState_Ensure();
/* Callback runs */
PyGILState_Release(cb_gilstate);
/* Python code and extension code unwinds */
PyGILState_Release(gilstate);
However, it *doesn't* work (at least, not easily) if the extension itself wants
to call back into an interpreter other than the one already associated with the
current thread:
/* Embedding application (GIL always unlocked) */
gilstate = PyGILState_EnsureEx(interp_A);
/* Python code and extension code happens */
/* Callback needs to reach back into a specific interpreter */
cb_gilstate = PyGILState_EnsureEx(interp_B);
/* Callback runs */
PyGILState_Release(cb_gilstate);
/* Python code and extension code unwinds */
PyGILState_Release(gilstate);
Does that second call to EnsureEx fail? If it succeeds, how does the client
know which interpreter to use for the PyGILState_Release call? It could be made
to work if PyGILState_STATE was changed from an enum to a struct that included
in interpreter state pointer, or if EnsureEx returned a different type and was
paired up with a new ReleaseEx pointer.
However, that's starting to get very messy compared to a separate
SwitchInterpreter call:
/* Embedding application (GIL always unlocked) */
old_interp = PyGILState_SwitchInterpreter(interp_A);
/* "autoTLSkey" now refers to a thread state for interpreter A */
gilstate = PyGILState_Ensure();
/* Python code and extension code happens */
/* Callback needs to reach back into Python */
pre_cb_interp = PyGILState_SwitchInterpreter(interp_B);
/* "autoTLSkey" now refers to a thread state for interpreter B */
cb_gilstate = PyGILState_Ensure();
/* Callback runs */
PyGILState_Release(cb_gilstate);
PyGILState_SwitchInterpreter(pre_cb_interp);
/* "autoTLSkey" again refers to a thread state for interpreter A */
/* Python code and extension code unwinds */
PyGILState_Release(gilstate);
PyGILState_SwitchInterpreter(old_interp);
And yes, I'm pondering ways that this could be used to implement Rust-style
"channels" for communication between interpreters without needing to copy data,
by using this API to create proxy interfaces for accessing an object owned by
another subinterpreter.
----------
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue15751>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
