On 18/05/07, Daniel J. Popowich <[EMAIL PROTECTED]> wrote:

Graham Dumpleton writes:
> And that is what the documentation I pointed you at states:
> """Note that with the new module importer, as directories associated
> with Python*Handler directives are no longer being added automatically
> to sys.path and they are instead used directly by the module importer
> only when required, some existing code which expected to be able to
> import modules in the handler root directory from a module in a
> subdirectory may no longer work. In these situations it will be
> necessary to set the mod_python module importer path to include '~' or
> list '~' in the __mp_path__ attribute of the module performing the
> import."""

OK, I think it's slowly sinking in...when I put the ~ in, it didn't
affect sys.path, so my import still wasn't going to work.  Right?

Correct, you were only changing the path related to modules managed by
the new module importer and not those handled by the standard Python
importer. Adding "~" like that would also only have been needed if in
a subdirectory of the handler root directory an import was trying to
be done with either 'import' or 'apache.import_module()' for a module
in the handler root directory.

I had assumed in suggesting that, that you had already converted
mpservlets to use the new module importer and were triggering that
specific case.

So, how the heck do I import the modules?

In just about all cases exactly how you do it now. Although, where you
were using your own module importing and caching system for working
under mod_python, you would need to adapt it to use the new importer.

Do I have to use apache.import_module() now?

No. In fact there are now cases where before you had to use
apache.import_module() because of bugs but you now can use just
'import' and it will also work, thus allowing one to create sets of
modules that can also be used safely outside of mod_python in a test
framework. That you have an existing module importing and caching
system of your own is a special case but only the module importer code
in moservlets should need to change, not anything else.

But I don't want to!

As far as using it in mpservlets itself, that would be a pity really
as one of the main reasons that support for importing files by full
path with an arbitrary extension was added was so that mpservlets
could make use of the new importer rather than use its own, and thus
benefit from some of the features the new module importer provided
such as root modules being imported automatically when a child was
changed thereby avoiding the need to restart Apache after anything but
simple changes. Thus, in the tutorial example, I could adjust
_SitePage and thereby notionally change the appearance of the whole
site without a restart, something that couldn't be avoided with how
mpservlets worked with previous versions of mod_python.

> If one wants to be precise, the addition of the handler directory to
> sys.path was actually done by the top level mod_python dispatch code
> rather than the module importer itself.

And this I don't understand at all.  If this "feature" was in the old
system, why would reimplementing an api function change this behaviour
in the dispatch code?  I would consider this a bug.

That it did add the handler directory to sys.path previously is what
caused a lot of the bugs documented in the page I referred to in the
first place. There was no choice but to change it else you couldn't
fix the existing bugs. You may have avoided the bugs by using your own
module importer, but the bugs still affected everyone else.

The whole point of the changes which were made was to draw a well
defined line between the code modules used in the web application and
which reside in the document tree, or other specially specified areas
by way of mod_python module importer path, and the standard Python
sys.path locations. That there was no distinction before is what
caused all the issues with same named modules, loading of incorrect
modules, reloading of modules which shouldn't be reloaded etc etc.

I would also say
this is WAY too much functionality change for a minor release update.

I did suggest that the release be called 4.0, but since there was a
backwards compatibility switch so people could reenable old behaviour
for module importer while they update their code the consensus was
that 3.3 was okay. Also, the whole thing was intended to be as
compatible as possible. In the end there was only a few cases where
people would actually have to make changes, the main one being where
people were relying on handler root direct being in sys.path so
imports from non mod_python managed modules could find a module in
that directory.

> The other important bit is:
> """As a new feature in mod_python 3.3, when using the standard Python
> 'import' statement to import a module, if the import is being done
> from a module which was previously imported by the mod_python module
> importer, it is equivalent to having called apache.import_module()
> directly."""
> It is because of this that the directory doesn't need to be in
> sys.path, as under the covers, when 'import' is being used by modules
> imported by the mod_python module importer directly in some way, then
> 'import' is equivalent to having used 'apache.import_module()' and
> therefore it looks in places related to mod_python including stuff in
> the separate mod_python importer path.

And I would call this a bug, too.  Why should using mod_python change
the behaviour of the import builtin?  What if I'm including a package
that includes a package that includes a package?  I'd have no idea
what code is doing the work.

Once a module is imported which isn't being managed by mod_python,
then the mapping of 'import' to 'apache.import_module()' doesn't get
triggered. Thus, it applies only to stuff in the web application and
not to anything on sys.path.

FWIW, overriding 'import' in ways like this is not uncommon in Python
as this is how importing stuff from Zip files and unexpanded Python
eggs works. The stuff works and it steps out of the way when it
shouldn't be applied.

I remember when I first brought up this idea of 'import' working in
magic ways so things would just work as one expected, that there were
at least one person who thought it was a silly idea. In the end I had
to give up on even trying to explain it because it would have taken me
months just to get people to understand what I had in mind. Instead, I
just wrote the code and I believe most would have to say the result in
certainly better than what was there before and that code now just
works as how one would expect it to work when one uses 'import'
whereas it didn't always before.

This is getting way too obfuscated for my simple ways.

All I can suggest is you try it by starting with the changes to
mpservlets as I described and then simply see if your existing
applications which use mpservlets still work. Since the release we
have only had a few people even be slightly affected by the
differences, plus all the complaints we used to get about problems
with the old importer have pretty well vanished. The only known issues
which currently exist are case sensitivity on Windows and a race
condition on imports causing a redundant reload. Both issues are
something that will be able to be fixed though.


Reply via email to