STINNER Victor <[email protected]> added the comment:
> test_atexit leaked [3988, 3986, 3988] references, sum=11962
The following patch fix it:
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
index 7591f069b4..f088ef0bce 100644
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -1210,6 +1210,15 @@ finalize_interp_clear(PyThreadState *tstate)
{
int is_main_interp = _Py_IsMainInterpreter(tstate);
+ _PyImport_Cleanup(tstate);
+
+ /* Explicitly break a reference cycle between the encodings module and XXX
*/
+ PyInterpreterState *interp = tstate->interp;
+ Py_CLEAR(interp->codec_search_path);
+ Py_CLEAR(interp->codec_search_cache);
+ Py_CLEAR(interp->codec_error_registry);
+ _PyGC_CollectNoFail();
+
/* Clear interpreter state and all thread states */
PyInterpreterState_Clear(tstate->interp);
@@ -1640,7 +1649,6 @@ Py_EndInterpreter(PyThreadState *tstate)
Py_FatalError("Py_EndInterpreter: not the last thread");
}
- _PyImport_Cleanup(tstate);
finalize_interp_clear(tstate);
finalize_interp_delete(tstate);
}
Py_NewInterpreter() indirectly calls "import encodings" which calls
codecs.register(search_function). This encodings function is stored in
interp->codec_search_path and so keeps encodings module dict alive.
_PyImport_Cleanup() removes the last reference to the encodings *module*, but
the module deallocator function (module_dealloc()) doesn't clear the dict: it
only removes its strong reference to it ("Py_XDECREF(m->md_dict);").
interp->codec_search_path is cleared by PyInterpreterState_Clear() which is
called by Py_EndInterpreter(). But it is not enough to clear some objets. I'm
not sure if encodings module dict is still alive at this point, but it seems
like at least the sys module dict is still alive.
I can push my workaround which manually "break a reference cycle" (really?
which one?), but I may be interested to dig into this issue to check if we
cannot find a better design.
_PyImport_Cleanup() and _PyModule_Clear() functions are fragile. They implement
smart heuristics to attempt to keep Python functional as long as possible *and*
try to clear everything. The intent is to be able to log warnings and
exceptions during the Python shutdown, for example.
The problem is that the heuristic keeps some objects alive longer than
expected. For example, I would expect that _PyImport_Cleanup() not only calls
sys.modules.clear(), but also clears the dict of each module
(module.__dict__.clear()). It doesn't, and I'm not sure why.
----------
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue36854>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
