Eric Snow <ericsnowcurren...@gmail.com> added the comment:
> I see multiple non trivial problems: > > * To finalize the subinterpreter A, Py_Finalize() may switch the current > Python thread state from the main interpreter to a Python thread in the > subintrepeter A. It can lead to new funny issues. I'm not sure what you mean with this. Are you talking about how another thread might be running that threadstate already? That doesn't sounds like a problem specific to this issue but rather a problem with Py_EndInterpreter() in general. > * A subinterpreter can be stuck for whatever reason and refuse to stop. For > example, the subinterpreter A may wait for an even from subinterpreter B. If > we don't run two interpreters "in parallel" (in threads), it may be stuck > forever. The wait_for_thread_shutdown() call in Py_FinalizeEx() already has a similar behavior. In the case of one interpreter blocking another like that, how would that be possible where it isn't already a problem? > * Python 3.10 still has weird code to allow daemon threads to continue to run > after Py_Finalize() complete. Maybe we need to add such hacks for > subinterpreter which fail to be finalized? For example, kill them (exit) as > soon as they attempt to acquire the GIL. Search for tstate_must_exit() in > Python/ceval_gil.h. I consider the problem of daemon threads to be a separate problem. The solution proposed here for finalization doesn't change any behavior regarding daemon threads. > By the way, currently Py_Finalize() calls PyInterpreterState_Clear() which > call object finalizers in the main thread of the main interpreter, whereas > these finalizers might expect to be called from the thread which created them. Do you have any examples? I'm having trouble imagining such a case. > I don't know if we must change anything else. > > Again, all these problems are very complex :-( I agree. However, automatically finalizing all other interpreters at the beginning of Py_FinalizeEx() doesn't introduce any new problems. At the same time, it makes sure that any resources in use in other interpreters have a chance to be released and that global resources used there don't cause crashes during runtime finalization. > The simple option, which is sadly a backward incompatible change, is to raise > a fatal error in Py_Finalize() if a subinterpreter is still running. As long as we require Py_FinalizeEx() to be called from the main interpreter, I don't see how the proposed change (in PR #17575) would be backward incompatible. > Maybe we can start by logging a warning into stderr for now, before > introducing the backward incompatible change. Maybe even only emit it when -X > dev is used? https://docs.python.org/dev/library/devmode.html I'd like to see a ResourceWarning if any other interpreters were still around. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue36225> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
