Re: New importer is now the default in 3.3 trunk. (PART 3)

[Graham Dumpleton]

On 20/08/2006, at 8:14 PM, Graham Dumpleton wrote: On 20/08/2006, at 11:22 AM, Graham Dumpleton wrote: FYI. Part 1 of some information about new module importer in 3.3. Part 2 of information about changes to new importer. And now for the final part 3. When using apache.import_module(), the old importer only allowed you to provide the name of a module or submodule in a package. Thus:   module1 = apache.import_module('module1')   module2 = apache.import_module('package1.module2') The new importer still allows this, except that the new importer will never itself load packages and instead relies on builtin importer of Python to do it. This means that packages must be somewhere on sys.path and are not candidates for module reloading. On some platforms (Linux but not MacOSX), you could also do use a slash in the name to get a module from a subdirectory. Ie.,   module3 = apache.import_module('prefix/module3') I don't actually know whether the top level had to be setup as a package with an __init__.py file or not. In the new importer, you can use a slash as well, without needing a __init__.py file in the subdirectory, and if it resides on the new module importers search path it will use it. With the new module importer, this becomes a simplified way of grouping modules together in a common directory, but where the common directory name is the only thing visible at top level scope. Ie., like a package but not a package. As long as imports using 'import' amongst modules in that common directory are local and don't try and reference back through the common directory root, they will work. Having mentioned 'import', an importer feature of the new module importer is that you can use 'import' in place of apache.import_module() for file module imports. The first place that will be searched for such modules is the same directory as the file doing the importer. Thus there is no need to set PythonPath to directories in the document tree like you had to with the old importer and in fact if you do, you will possibly get a nasty warning in the error logs about setting PythonPath to a directory which is actually a directory managed by the new importer. Note that 'import' will only use the new importer in this way if the module doing the import was itself imported using the new module importer. Thus, if the module was on sys.path somewhere, imports from those modules work like before and don't use the new module importer. In respect of 'import' looking in the same directory first, apache.import_module() will also do this. When specifying just a module name to apache.import_module() or with the 'import' statement where applicable, the module will be searched along a path consisting of current directory, embedded __mp_path__ of that module and value of mod_python.importer.path option. One can also specify other directories to search for a specific import using apache.import_module() by specifying them using the 'path' argument like with the old importer.   module5 = apache.import_module('module5', path=['/some/path/modules']) This will just be an additional place which is looked. If is necessary to specify exactly where the module resides, the new module importer now allows a full path to the module to be specified.   module6 = apache.import_module('/some/path/modules/_module6.py') The extension must be specified, and it doesn't actually have to be .py. the mpservlets package for example might be updated to use the new importer and use:   module7 = apache.import_module('/some/path/module7.mps') This gives some certainty about which module is imported. Note that when specifying just the module name, the file must always have a .py extension. Rather than specifying a full path, it is also possible to specify a relative path. This will be evaluated relative to the directory the file doing the import is located in.   module8 = apache.import_module('./module8.py')   module9 = apache.import_module('../module9.py')   module10 = apache.import_module('../modules/module10.py') An odd little thing is that when a full or relative path name is supplied, any list of directories specified with the 'path' argument to apache.import_module() as the starting value of the embedded path __mp_path__. This was originally done to support generated code from Cheetah templates, but not strictly be required now given that mod_python.importer.path option exists. All of the above already worked and existed in the new module importer. With the last set of changes which made the new module importer the default, an old feature was also resurrected. That is, one can also specify as a path to get a module from the handler root directly, using '~/' prefix to path. Thus:   module11 = apache.import_module('~/module11.py') This is a short cut so that the handler root doesn't have to be set as an embedded path using __mp_path__ or set explicitly in mod_python.importer.path. BTW, the 'log' and 'debug' arguments to apache.import_module() are effectively redundant now although they are still there. The arguments default to None which means the importer will pick up the values itself by accessing the config table object using apache.get_current_config(). Because these arguments are 2nd and 3rd and the 'path' is 4th, you should always use a keyword argument for 'path' and simply skip 'log' and 'debug'. The 'log' and 'debug' arguments are kept for backward compatibility only. Okay, that is it for now. I'll have more to say about mod_python.importer.path in a day or so when what I have posted so far has been digested. Because the value of mod_python.importer.path is eval'd there are some cute backdoor tricks that can be used to avoiding using absolute path names in it and these tricks should perhaps be formalised and agreed that they be allowed. Until next time. I'll stop bombarding you will mail now for the rest of my weekend at least. :-) Graham