Nick Coghlan added the comment: The reason I'm proposing going back to the original SwitchInterpreter idea is because the EnsureEx idea doesn't nest cleanly - if your thread is already associated with "interpreter A", you can't readily call into "interpeter B", because the API provides no way to correctly restore the associated interpreter back to interpreter A when you're done.
EnsureEx works fine if extension modules are not aware of multiple interpreters: /* Embedding application (GIL always unlocked) */ gilstate = PyGILState_EnsureEx(interp_A); /* Python code and extension code happens */ /* Callback needs to reach back into Python */ cb_gilstate = PyGILState_Ensure(); /* Callback runs */ PyGILState_Release(cb_gilstate); /* Python code and extension code unwinds */ PyGILState_Release(gilstate); However, it *doesn't* work (at least, not easily) if the extension itself wants to call back into an interpreter other than the one already associated with the current thread: /* Embedding application (GIL always unlocked) */ gilstate = PyGILState_EnsureEx(interp_A); /* Python code and extension code happens */ /* Callback needs to reach back into a specific interpreter */ cb_gilstate = PyGILState_EnsureEx(interp_B); /* Callback runs */ PyGILState_Release(cb_gilstate); /* Python code and extension code unwinds */ PyGILState_Release(gilstate); Does that second call to EnsureEx fail? If it succeeds, how does the client know which interpreter to use for the PyGILState_Release call? It could be made to work if PyGILState_STATE was changed from an enum to a struct that included in interpreter state pointer, or if EnsureEx returned a different type and was paired up with a new ReleaseEx pointer. However, that's starting to get very messy compared to a separate SwitchInterpreter call: /* Embedding application (GIL always unlocked) */ old_interp = PyGILState_SwitchInterpreter(interp_A); /* "autoTLSkey" now refers to a thread state for interpreter A */ gilstate = PyGILState_Ensure(); /* Python code and extension code happens */ /* Callback needs to reach back into Python */ pre_cb_interp = PyGILState_SwitchInterpreter(interp_B); /* "autoTLSkey" now refers to a thread state for interpreter B */ cb_gilstate = PyGILState_Ensure(); /* Callback runs */ PyGILState_Release(cb_gilstate); PyGILState_SwitchInterpreter(pre_cb_interp); /* "autoTLSkey" again refers to a thread state for interpreter A */ /* Python code and extension code unwinds */ PyGILState_Release(gilstate); PyGILState_SwitchInterpreter(old_interp); And yes, I'm pondering ways that this could be used to implement Rust-style "channels" for communication between interpreters without needing to copy data, by using this API to create proxy interfaces for accessing an object owned by another subinterpreter. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue15751> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com