[issue24554] GC should happen when a subinterpreter is destroyed

STINNER Victor added the comment: Py_EndInterpreter() now calls gc.collect() at least twice: at the end of _PyImport_Cleanup() and in finalize_interp_clear(). I now consider the issue as fixed and so I close it. The issue that I described in my previous comment can be enhanced/fixed later.

[issue24554] GC should happen when a subinterpreter is destroyed

STINNER Victor added the comment: This issue is partially fixed in the master branch. Extract of the finalize_interp_clear() function, called by Py_EndInterpreter(): /* Clear interpreter state and all thread states */ PyInterpreterState_Clear(tstate->interp); /* Trigger a GC

[issue24554] GC should happen when a subinterpreter is destroyed

STINNER Victor added the comment: bpo-36854 has been fixed, so it's time to reconsider fixing this issue :-) -- nosy: +vstinner ___ Python tracker ___

[issue24554] GC should happen when a subinterpreter is destroyed

Change by Paulo Henrique Silva : -- nosy: +phsilva ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue24554] GC should happen when a subinterpreter is destroyed

Eric Snow added the comment: FYI, issue #36854 is about moving GC runtime state from _PyRuntimeState to PyInterpreterState. However, that doesn't trigger any collection when the interpreter is finalized. So there is more to be done here. -- ___

[issue24554] GC should happen when a subinterpreter is destroyed

Graham Dumpleton added the comment: That GC happens on an object in the wrong interpreter in this case is the problem as it can result in used code execution against the wrong interpreter context. If you are saying this can happen anytime in the life of a sub interpreter and not just in this

[issue24554] GC should happen when a subinterpreter is destroyed

Antoine Pitrou added the comment: It's just a consequence of subinterpreters not being isolated contexts. They're sharing of lot of stuff by construction (hence being called subinterpreters). And indeed some resource can become unreachable in a subinterpreter, and collected from another, if

[issue24554] GC should happen when a subinterpreter is destroyed

Antoine Pitrou added the comment: I don't think this is a very important issue, by the way. Normal destructors will usually rely on resources on their global environment, i.e. the function's globals or builtins dict, which will point to the right namespace. Only if you are explicitly looking

[issue24554] GC should happen when a subinterpreter is destroyed

Antoine Pitrou added the comment: From a quick look at the PyInterpreterState, stuff that may be risky to rely on: - mutable data from the sys module (mainly import-related data: sys.path, sys.meta_path, etc.) - codecs registry metadata Of course third-party modules (C extensions) may key

[issue24554] GC should happen when a subinterpreter is destroyed

Graham Dumpleton added the comment: If this issue with GC can't be addressed and sub interpreters isolated better, then there is no point pursing then the idea that has been raised at the language summit of giving each sub interpreter its own GIL and then provide mechanisms to allow code

[issue24554] GC should happen when a subinterpreter is destroyed

Antoine Pitrou added the comment: What is that supposed to demonstrate? GC is a global operation and is not tied to subinterpreters: GC may happen in any interpreter, not necessarily the one where the resource was allocated. -- nosy: +pitrou ___

[issue24554] GC should happen when a subinterpreter is destroyed

Antoine Pitrou added the comment: I don't have any opinion on whether this issue kills the parallelism with sub-interpreters idea (I'm not sure why it would). But regardless, solving this issue will be very non-trivial, or perhaps very involved. Running the GC at the end of a subinterpreter

[issue24554] GC should happen when a subinterpreter is destroyed

Graham Dumpleton added the comment: Right now mod_wsgi is the main user of sub interpreters. I wasn't even aware of this issue until Jesse found it. Thus in 7+ years, it never presented a problem in practice, possibly because in mod_wsgi sub interpreters are only ever destroyed on process

[issue24554] GC should happen when a subinterpreter is destroyed

Nick Coghlan added the comment: We already knew that reference count management was likely to be one of the thorniest problems with allowing true subinterpreter level concurrency, this issue just reminds us that the cyclic GC is going to be a challenge as well. --

[issue24554] GC should happen when a subinterpreter is destroyed

New submission from Eric Snow: Per A. Jesse Jiryu Davis: === # mod.py class C(object): pass class Pool(object): def __del__(self): print('del') list() C.pool = Pool() === === int main() {