On Aug 12, 2011, at 11:24 AM, P.J. Eby wrote:
> That is, the above code hardocdes a variety of assumptions about the import
> system that haven't been true since Python 2.3.
Thanks for this feedback. I honestly did not realize how old and creaky this
code had gotten. It was originally developed for Python 2.4 and it certainly
shows its age. Practically speaking, the code is correct for the bundled
importers, and paths and zipfiles are all we've cared about thus far.
> (For example, it assumes that the contents of sys.path strings have
> inspectable semantics, that the contents of __file__ can tell you things
> about the module-ness or package-ness of a module object, etc.)
Unfortunately, the primary goal of this code is to do something impossible -
walk the module hierarchy without importing any code. So some heuristics are
necessary. Upon further reflection, PEP 402 _will_ make dealing with namespace
packages from this code considerably easier: we won't need to do AST analysis
to look for a __path__ attribute or anything gross like that improve
correctness; we can just look in various directories on sys.path and accurately
predict what __path__ will be synthesized to be.
However, the isPackage() method can and should be looking at the module if it's
already loaded, and not always guessing based on paths. The whole reason
there's an 'importPackages' flag to walk() is that some applications of this
code care more about accuracy than others, so it tries to be as correct as it
can be.
(Of course this is still wrong for the case where a __path__ is dynamically
constructed by user code, but there's only so well one can do at that.)
> If you want to fully support PEP 302, you might want to consider making this
> a wrapper over the corresponding pkgutil APIs (available since Python 2.5)
> that do roughly the same things, but which delegate all path string
> inspection to importer objects and allow extensible delegation for importers
> that don't support the optional methods involved.
This code still needs to support Python 2.4, but I will make a note of this for
future reference.
> (Of course, if the pkgutil APIs are missing something you need, perhaps you
> could propose additions.)
>> Now it seems like pure virtual packages are going to introduce a new type of
>> special case into the hierarchy which have neither .pathEntry nor .filePath
>> objects.
>
> The problem is that your API's notion that these things exist as coherent
> concepts was never really a valid assumption in the first place. .pth files
> and namespace packages already meant that the idea of a package coming from a
> single path entry made no sense. And namespace packages installed by
> setuptools' system packaging mode *don't have a __file__ attribute* today...
> heck they don't have __init__ modules, either.
The fact that getModule('sys') breaks is reason enough to re-visit some of
these design decisions.
> So, adding virtual packages isn't actually going to change anything, except
> perhaps by making these scenarios more common.
In that case, I guess it's a good thing; these bugs should be dealt with.
Thanks for pointing them out. My opinion of PEP 402 has been completely
reversed - although I'd still like to see a section about the module system
from a library/tools author point of view rather than a time-traveling perl
user's narrative :).
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
