New submission from pkerling <9b6ab...@casix.org>: The C API documentation says this about single-phase initialization and sub-interpreters:
"[T]he first time a particular extension is imported, it is initialized normally, and a (shallow) copy of its module’s dictionary is squirreled away. When the same extension is imported by another (sub-)interpreter, a new module is initialized and filled with the contents of this copy; the extension’s init function is not called." - from https://docs.python.org/3.10/c-api/init.html#c.Py_NewInterpreter I was investigating crashes relating to the _datetime module and sub-interpreters and from my observations, this does not seem to be true. I have tracked this functionality down to the m_base.m_copy dictionary of the PyModuleDef of an extension and the functions _PyImport_FixupExtensionObject and _PyImport_FindExtensionObject in Python/import.c. However, modules are only ever added to the `extensions` global when imported in the main interpreter, see https://github.com/python/cpython/blob/1f73c320e2921605c4963e202f6bdac1ef18f2ce/Python/import.c#L480 Furthermore, even if they were added and m_base.m_copy was set, it would be cleared again on sub-interpreter shutdown here: https://github.com/python/cpython/blob/1f73c320e2921605c4963e202f6bdac1ef18f2ce/Python/pystate.c#L796 - implying that the module will be loaded and initialized again next time due to this check: https://github.com/python/cpython/blob/1f73c320e2921605c4963e202f6bdac1ef18f2ce/Python/import.c#L556 These observations are supported by the fact that in my tests, if "import _datetime" is ran subsequently in two different sub-interpreters, PyInit__datetime is indeed called twice. Test code - set a breakpoint on PyInit__datetime to observe the behavior: #include <Python.h> #include <assert.h> int main() { Py_Initialize(); for (int i = 0; i < 100; ++i) { PyThreadState* ts = Py_NewInterpreter(); assert(ts); int result = PyRun_SimpleString("import _datetime"); assert(result == 0); Py_EndInterpreter(ts); } return 0; } In summary, it seems to me that the documented behavior is not accurate (any more?) - so either the docs or the implementation should change. ---------- assignee: docs@python components: Documentation messages: 380602 nosy: docs@python, pkerling priority: normal severity: normal status: open title: Documented interaction of single-stage init and sub-interpreters inaccurate versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue42298> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
