On 1 September 2013 23:03, Antoine Pitrou <solip...@pitrou.net> wrote: > On Sun, 1 Sep 2013 11:28:36 +1000 > Nick Coghlan <ncogh...@gmail.com> wrote: >> * PEP 3121 with a size of "0". As above, but avoids the module state APIs >> in order to support reloading. All module state (including type >> cross-references) is stored in hidden state (e.g. an instance of a custom >> type not exposed to Python, with a reference stored on each custom type >> object defined in the module, and any module level "functions" actually >> being methods of a hidden object). Still doesn't support loading a *fresh* >> copy due to the hidden PEP 3121 module cache. > > Not sure what you mean by that: > >>>> import atexit >>>> id(atexit) > 140031896222680 >>>> import sys >>>> del sys.modules['atexit'] >>>> import atexit >>>> id(atexit) > 140031896221400
Ah, you're right - I misremembered the exact problem that broke xml.etree.ElementTree testing. PyModule_GetState is actually fine (since that pointer is hidden state on the module object), it's only PyState_GetModule that is broken when you import a second copy. So, here, when the second import happens, it breaks the original atexit module's callbacks, even though the two callback registries are properly isolated: $ ./python Python 3.4.0a1+ (default:575071257c92+, Aug 25 2013, 00:42:17) [GCC 4.7.2 20121109 (Red Hat 4.7.2-8)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import atexit >>> atexit.register(print, "Hello World!") <built-in function print> >>> import sys >>> del sys.modules["atexit"] >>> import atexit as atexit2 >>> atexit2.register(print, "Goodbye World!") <built-in function print> >>> Goodbye World! So I think PEP 3121 is actually as good as we can get on the hidden state front, but the important point is that it is the *PyState_GetModule* API that can't handle fresh imports - the second import will always replace the first one. So anyone affected needs to find some other way of passing the state, like using bound methods of a hidden type rather than ordinary callables. If you have to interoperate with a C API that only accepts a C callback without allowing additional state arguments, you're going to have trouble. I think atexit serves as a good example, though - that _Py_PyAtExit call will *always* be destructive (even if you still have a reference to the original module), so there should be a way for the module to explicitly indicate to the import system "you can only create this module once, and then you're committed - unloading it and importing it again won't work properly due to side effects on the process state". Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
