On 25 Aug 2013 01:44, "Stefan Behnel" <stefan...@behnel.de> wrote: > > Nick Coghlan, 24.08.2013 16:22: > > On 24 August 2013 23:19, Stefan Behnel wrote: > >> Nick Coghlan, 24.08.2013 13:36: > >>> On 24 August 2013 15:51, Nick Coghlan wrote: > >>>> My current plan is to create an experimental prototype of this > >>>> approach this weekend. That will include stdlib test cases, so it will > >>>> also show how it looks from the extension developer's point of view. > >>> > >>> I prototyped as much as I could without PEP 451's ModuleSpec support here: > >>> > >>> https://bitbucket.org/ncoghlan/cpython_sandbox/commits/branch/new_extension_imports > >> > >> Cool. I'll take a look. > > > > The new _PyImport_CreateAndExecExtensionModule function does the heavy lifting: > > > > https://bitbucket.org/ncoghlan/cpython_sandbox/src/081f8f7e3ee27dc309463b48e6c67cf4880fca12/Python/importdl.c?at=new_extension_imports#cl-65 > > > > One key point to note is that it *doesn't* call > > _PyImport_FixupExtensionObject, which is the API that handles all the > > PEP 3121 per-module state stuff. Instead, the idea will be for modules > > that don't need additional C level state to just implement > > PyImportExec_NAME, while those that *do* need C level state implement > > PyImportCreate_NAME and return a custom object (which may or may not > > be a module subtype). > > Is it really a common case for an extension module not to need any C level > state at all? I mean, this might work for very simple accelerator modules > with only a few stand-alone functions. But anything non-trivial will almost > certainly have some kind of global state, cache, external library, etc., > and that state is best stored at the C level for safety reasons.
I'd prefer to encourage people to put that state on an exported *type* rather than directly in the module global state. So while I agree we need to *support* C level module globals, I'd prefer to provide a simpler alternative that avoids them. We also need the create/exec split to properly support reloading. Reload *must* reinitialize the object already in sys.modules instead of inserting a different object or it completely misses the point of reloading modules over deleting and reimporting them (i.e. implicitly affecting the references from other modules that imported the original object). > > Such modules can still support reloading (e.g. > > to pick up reloaded or removed module dependencies) by providing > > PyImportExec_NAME as well. > > > > (in a PEP 451 world, this would likely be split up as two separate > > functions, one for create, one for exec) > > Can't we just always require extension modules to implement their own type? > Sure, it's a lot of boiler plate code, but that could be handled by a > simple C code generator or maybe even a copy&paste example in the docs. I > would like to avoid making it too easy for users in the future to get > anything wrong with reloading or sub-interpreters. Most people won't test > these things for their own code and the harder it is to make them not work, > the more likely it is that a given set of dependencies will properly work > in a sub-interpreter. > > If users are required to implement their own type, I think it would be more > obvious where to put global module state, how to define functions (i.e. > module methods), how to handle garbage collection at the global module > level, etc. Take a look at the current example - everything gets stored in the module dict for the simple case with no C level global state. The module level functions are still added through a Py_MethodDef array, the docstring still comes from a C char pointer. I did have to fix the custom type's tp_new method to use the type pointer passed in by the interpreter rather than a C static global pointer, but that change would also have been needed if defining a custom type. Since Antoine fixed it, there's also nothing particularly quirky about module destruction in 3.4+ - cyclic GC should "just work". Cheers, Nick.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
