[issue42671] Make the Python finalization more deterministic

STINNER Victor added the comment: I'm not really excited about breaking applications. While there is maybe a way to make the Python finalization more determistic, sorry, I'm not the one who will be brave enough to do it today. I abandon my idea for now. -- resolution: -> rejected st

[issue42671] Make the Python finalization more deterministic

STINNER Victor added the comment: > Drawback: it is a backward incompatible change. Code which worked by luck > previously no longer works. I'm talking about applications which rely on > __del__() methods being calling in an exact order and expect Python being in > a specific state. Python

[issue42671] Make the Python finalization more deterministic

STINNER Victor added the comment: > I wrote this issue to attempt to fix a leak in PR 23811 which converts the > _thread extension module to the multiphase initialization API (PEP 489). I fixed this bug by adding a traverse function to the _thread lock type. So I don't *need* this issue to u

[issue42671] Make the Python finalization more deterministic

Serhiy Storchaka added the comment: I experimented with different approaches last evening. Just changing the order of clearing the module dict does not help your first example, because the __main__ module is garbage collected, but its dict is not cleared. Adding _PyModule_ClearDict() in tp_d

[issue42671] Make the Python finalization more deterministic

STINNER Victor added the comment: I took some notes on latest changes of the Python finalization order: https://pythondev.readthedocs.io/finalization.html#reorder-python-finalization Not directly related, I also took notes on the "weird GC behavior during Python finalization" (such as the iss

[issue42671] Make the Python finalization more deterministic

STINNER Victor added the comment: > Modules are already cleared in the reverse order. In Python 3.9, _PyImport_Cleanup() of Python/import.c contains the comment: /* Since dict is ordered in CPython 3.6+, modules are saved in importing order. First clear modules imported la

[issue42671] Make the Python finalization more deterministic

Serhiy Storchaka added the comment: Modules are already cleared in the reverse order. Clearing module attributes in the reverse order looks interesting. It increases chance that modules and other imported names which can be used in __del__ will be deleted after executing __del__ if the modul

[issue42671] Make the Python finalization more deterministic

STINNER Victor added the comment: I wrote this issue to attempt to fix a leak in PR 23811 which converts the _thread extension module to the multiphase initialization API (PEP 489). -- ___ Python tracker __

[issue42671] Make the Python finalization more deterministic

Change by STINNER Victor : -- keywords: +patch pull_requests: +22688 stage: -> patch review pull_request: https://github.com/python/cpython/pull/23826 ___ Python tracker ___ _

[issue42671] Make the Python finalization more deterministic

New submission from STINNER Victor : At exit, Python calls Py_Finalize() which tries to clear every single Python objects. The order is which Python objects are cleared is not fully deterministic. Py_Finalize() uses an heuristic to attempt to clear modules of sys.modules in the "best" order.