Antoine Pitrou wrote: > In any case, the amount of disagreement and/or misunderstanding in this > discussion is a strong hint that it needs a PEP to hash things out and > explain them clearly, IMHO.
Agreed; a PEP (even if it's just informational) would go a long way in helping to clear up some misunderstandings. Perhaps something along the lines of "Supporting concurrent subinterpreters in the C-API" that provides both a high-level overview and low-level information on some of the different structures involved and any needed changes [1], such as to PyThreadState, PyInterpreterState, PyGILState_STATE, etc. Optimally, it would explain some changes made and needed changes to their public and private APIs, as well as to the struct members. I'd be happy to help with something like this (particularly in reviewing and providing feedback on any unclear parts), but my own current understanding is likely not strong enough to lead the efforts. In fact, I very likely am in the camp of having some significant misunderstandings about the low-level implementation details. My experience with subinterpreters [2] is mostly limited to attempting to help with some tricky bugs. I'd call it an "area of interest", but it's very far from an "area of expertise". ;-) In the meantime for anyone looking for a basic overview on thread state and interpreter state, there's some general information documented in https://docs.python.org/3.9/c-api/init.html#high-level-api. --- [1] - By "needed changes", I am specifically referring to any changes to the C-API or internals that were made or are still needed to support concurrent subinterpreters. [2] - I have much more experience in the existing areas of concurrency in the standard library, such as with asyncio and concurrent.futures. Although it's relevant, it's very different in terms to implementation details. Also, I'm substantially more interested in the Python parts of the stdlib rather than the C internals or extension modules, but I certainly have an interest in learning more. Regards, Kyle Stanley On Wed, Mar 18, 2020 at 11:53 AM Antoine Pitrou <solip...@pitrou.net> wrote: > On Wed, 18 Mar 2020 13:35:16 +0000 > Anders Munch <a...@flonidan.dk> wrote: > > Chris Angelico [mailto:ros...@gmail.com]: > > > And by that logic, globals are even more capable. I don't understand > your > > > point. Isn't the purpose of the tstate parameters to avoid the problem > of > > > being unable to have multiple tstates within the same OS thread? I > think I've > > > missed something here. > > > > The point is that because threads can't preempt themselves, this: > > > > /*1*/ > > { > > Py_something(other_tstate); > > } > > > > and this: > > > > /*2*/ > > { > > PyInterpreterState* old_tstate = tstate; > > tstate = other_state; > > Py_something(); > > tstate = old_tstate; > > } > > > > is effectively equivalent, provided tstate is thread-local. Both will > work > > equally well from the hypothetical C callback that wants to use a > different > > subinterpreter. > > This example is mixing up the notion of interpreter state and thread > state. > > Here is a more realistic example: > > struct UserData { > PyInterpreterState* interp; > }; > > // Callback given to a third-party C library > void my_c_callback(void* opaque) { > struct UserData* user_data = (struct UserData*) opaque; > PyGILState_STATE gstate; > gstate = PyInterpreter_GILStateEnsure(user_data->interp); > > // ... > > PyInterpreter_GILStateEnsure(user_data->interp, gstate); > } > > > Of course this implies the existence of a PyInterpreter_GILState API > that currently doesn't exist. In this model, the "thread-local" thread > state is a *per-interpreter* thread-local. A *process-wide* > thread-local thread state cannot work if we want -- in some distant > future -- to be able to run several subinterpreters concurrently. > > > In any case, the amount of disagreement and/or misunderstanding in this > discussion is a strong hint that it needs a PEP to hash things out and > explain them clearly, IMHO. > > Regards > > Antoine. > > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/EKZJYO75QBDKZA4A2RR43PKDYEJRD7HC/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T4EC2LXRCGTIN222HNILLVKUSAOZ3BGA/ Code of Conduct: http://python.org/psf/codeofconduct/
