Nick Coghlan, 24.08.2013 16:22: > On 24 August 2013 23:19, Stefan Behnel wrote: >> Nick Coghlan, 24.08.2013 13:36: >>> On 24 August 2013 15:51, Nick Coghlan wrote: >>>> My current plan is to create an experimental prototype of this >>>> approach this weekend. That will include stdlib test cases, so it will >>>> also show how it looks from the extension developer's point of view. >>> >>> I prototyped as much as I could without PEP 451's ModuleSpec support here: >>> >>> >>> https://bitbucket.org/ncoghlan/cpython_sandbox/commits/branch/new_extension_imports >> >> Cool. I'll take a look. > > The new _PyImport_CreateAndExecExtensionModule function does the heavy > lifting: > > > https://bitbucket.org/ncoghlan/cpython_sandbox/src/081f8f7e3ee27dc309463b48e6c67cf4880fca12/Python/importdl.c?at=new_extension_imports#cl-65 > > One key point to note is that it *doesn't* call > _PyImport_FixupExtensionObject, which is the API that handles all the > PEP 3121 per-module state stuff. Instead, the idea will be for modules > that don't need additional C level state to just implement > PyImportExec_NAME, while those that *do* need C level state implement > PyImportCreate_NAME and return a custom object (which may or may not > be a module subtype).
Is it really a common case for an extension module not to need any C level state at all? I mean, this might work for very simple accelerator modules with only a few stand-alone functions. But anything non-trivial will almost certainly have some kind of global state, cache, external library, etc., and that state is best stored at the C level for safety reasons. > Such modules can still support reloading (e.g. > to pick up reloaded or removed module dependencies) by providing > PyImportExec_NAME as well. > > (in a PEP 451 world, this would likely be split up as two separate > functions, one for create, one for exec) Can't we just always require extension modules to implement their own type? Sure, it's a lot of boiler plate code, but that could be handled by a simple C code generator or maybe even a copy&paste example in the docs. I would like to avoid making it too easy for users in the future to get anything wrong with reloading or sub-interpreters. Most people won't test these things for their own code and the harder it is to make them not work, the more likely it is that a given set of dependencies will properly work in a sub-interpreter. If users are required to implement their own type, I think it would be more obvious where to put global module state, how to define functions (i.e. module methods), how to handle garbage collection at the global module level, etc. >>> On systems that use dynload_shlib (at least Linux & the BSDs), this >>> branch allows extension modules to be imported if they provide a >>> PyImportExec_NAME hook. The new hook is preferred to the existing >>> PyInit_NAME hook, so extension modules using the stable ABI can >>> provide both and degrade to the legacy initialisation API on older >>> versions of Python. >> >> Hmm, right, good call. Since both init schemes have to be part of the >> stable ABI, we can's rely on people compiling out one or the other. So >> using the old one as a fallback should work. However, only actual usage in >> code will tell us how it feels on user side. Supporting both in the same >> binary will most likely complicate things quite a bit. > > It shouldn't be too bad - the PyInit_NAME fallback would just need to > do the equivalent of calling PyImportCreate_NAME (or PyModule_Create > if not using a custom object), call PyImportExec_NAME on it, and then > return the result. > > Modules that genuinely *needed* the new behaviour wouldn't be able to > provide a sensible fallback, and would thus be limited to Python 3.4+ Right. I only saw it from the POV of Cython, which *will* have to support both, and *will* use the new feature in Py3.4+. No idea how that is going to work, but we've found so many tricks and work-arounds in the past that I'm sure it'll work somehow. Module level properties are just way too tempting not to make use of them. Stefan _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
