At 03:46 AM 4/16/2009 +0000, gl...@divmod.com wrote:
On 15 Apr, 09:11 pm, p...@telecommunity.com wrote:
I think that there is some confusion here. A "main" package or
buildout that assembles a larger project from components is not the
same thing as having a "base" package for a namespace package.
I'm certainly confused.
Twisted has its own system for "namespace" packages, and I'm not
really sure where we fall in this discussion. I haven't been able
to follow the whole thread, but my original understanding was that
the PEP supports "defining packages", which we now seem to be
calling "base packages", just fine.
Yes, it does. The discussion since the original proposal, however,
has been dominated by MAL's counterproposal, which *requires* a
defining package.
There is a slight distinction between "base package" and "defining
package", although I suppose I've been using them a bit interchangeably.
Base package describes a use case: you have a base package which is
extended in the same namespace. In that use case, you may want to
place your base package in the defining package.
In contrast, setuptools does not support a defining package, so if
you have a base package, you must place it in a submodule or
subpackage of the namespace.
Does that all make sense now?
MAL's proposal requires a defining package, which is
counterproductive if you have a pure package with no base, since it
now requires you to create an additional project on PyPI just to hold
your defining package.
I'd appreciate it if the PEP could also be extended cover Twisted's
very similar mechanism for namespace packages,
"twisted.plugin.pluginPackagePaths". I know this is not quite as
widely used as setuptools' namespace package support, but its
existence belies a need for standardization.
The PEP also seems a bit vague with regard to the treatment of other
directories containing __init__.py and *.pkg files.
Do you have a clarification to suggest? My understanding (probably a
projection) is that to be a nested namespace package, you have to
have a parent namespace package.
The concept of a "defining package" seems important to avoid
conflicts like this one:
http://twistedmatrix.com/trac/ticket/2339
More specifically I don't quite understand the PEP's intentions
towards hierarchical packages. It says that all of sys.path will be
searched, but what about this case?
In Twisted, the suggested idiom to structure a project which wants
to provide Twisted plugins is to have a directory structure like this:
MyProject/
myproject/
__init__.py
twisted/
plugins/
myproject_plugin.py
If you then put MyProject on PYTHONPATH, MyProject/twisted/plugins
will be picked up automatically by the plugin machinery.
Namespaces are not plugins and vice versa. The purpose of a
namespace package is to allow projects managed by the same entity to
share a namespace (ala Java "package" names) and avoid naming
conflicts with other authors.
A plugin system, by contrast, is explicitly intended for use by
multiple authors, so the use case is rather different... and using
namespace packages for plugins actually *increases* the possibility
of naming conflicts, unless you add back in another level of
hierarchy. (As apparently you are recommending via "myproject_plugin".)
However, as "twisted" is *not* a "namespace" package in the same
way, .py files in MyProject/twisted/ would not be picked up - this
is very much intentional, since the "twisted" namespace is intended
to be reserved for packages that we actually produce. If either
MyProject/twisted or MyProject/twisted/plugins/ had an __init__.py,
then no modules in MyProject/twisted/plugins/ would be picked up,
because it would be considered a conflicting package.
Precisely. Note, however, that neither is twisted.plugins a
namespace package, and it should not contain any .pkg files. I don't
think it's reasonable to abuse PEP 382 namespace packages as a plugin
system. In setuptools' case, a different mechanism is provided for
locating plugin code, and of course Twisted already has its own
system for the same thing. It would be nice to have a standardized
way of locating plugins in the stdlib, but that will need to be a
different PEP.
I hope this all makes sense. As I understand it, both setuptools
and the proposed standard would either still have the bug described
by ticket 2339 above, or would ignore twisted/plugins/ as a
namespace package because its parent isn't a namespace package.
If twisted/ lacked an __init__.py, then setuptools would ignore
it. Under PEP 382, the same, unless it had .pkg files. (Again,
setuptools explicitly does not support using namespace packages as a
plugin mechanism.)
P.S.: vendor packaging systems *ARE* a major use case for just about
any aspect of Python's package structure. I really liked MvL's
coverage of "vendor packages", in the PEP, since this could equally
well apply to MSIs, python libraries distributed in bundles on OS X,
debs, or RPMs. If this use-case were to be ignored, as one
particular fellow seems to be advocating, then the broken packages
and user confusion that has been going on for the last 5 years or so
is just going to get worse.
Indeed.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
