Hello again,since I haven't got any replies yet I'm trying to make myself a bit more precise now. I consider the behaviour described in my original posting a bug. I posted to this list because the setuptools docs say "Please use the distutils-sig mailing list [3] for questions and discussion about setuptools, and the setuptools bug tracker [4] ONLY for issues you have confirmed via the list".
I have two questions, which I hope some expert here can answer:1) How do I properly handle PEP420 namespace packages with setuptools and pip (Python >= 3.3) assuming the scenario described below? 2) Is the behaviour of setuptools/pip as I encountered it intended (including the solution I found) or is it a bug that should be filed?
So here is, with some more detail, what I am trying to do: I am using Python 3.4. There are two projects with modules in the namaspace coolpkg. These implement the sub-packages coolpkg.bar and and coolpkg.foo respectively. Both projects have (as allowed by PEP420) no coolpkg/__init__.py. Both projects have a setup.py using setuptools (15.0) to create a source distribution and will be installed using pip (6.1.1).
In addition to that, there is another submodule coolpkg.baz, which will not be packaged/installed using setuptools/pip. Instead the folder containing it, will we added to the PYTHONPATH.
Here is the complete layout of the projects and the additional module. project-bar/ project-bar/setup.py project-bar/src project-bar/src/coolpkg project-bar/src/coolpkg/bar project-bar/src/coolpkg/bar/__init__.py project-bar/src/coolpkg/bar/barmod.py project-foo/ project-foo/setup.py project-foo/src project-foo/src/coolpkg project-foo/src/coolpkg/foo project-foo/src/coolpkg/foo/foomod.py project-foo/src/coolpkg/foo/__init__.py shady-folder/ shady-folder/coolpkg shady-folder/coolpkg/baz shady-folder/coolpkg/baz/__init__.py shady-folder/coolpkg/baz/bazmod.pyMy goal is to have a runtime configuration such that modules coolpkg.foo.foomod, coolpkg.bar.barmod, coolpkg.baz.bazmod can all be imported.
Test 1) (just a basic test to verify general setup)Add project-bar/src, project-foo/src and shady-folder manually to sys.path
Result: works (obviously) For further tests:Create source distributions with setup.py sdist for project-bar and project-foo, install them with pip and put shady-folder on PYTHONPATH. Declare packages=['coolpkg', 'coopkg.foo'] and packages=['coolpkg', 'coopkg.bar'] in respectice setup.py, since find_packages does not play well with PEP420.
Test 2) Delcare namespace_packages=['coolpkg'] in setup.py of each projectResult: coolpkg.foo.foomod and coolpkg.bar.barmod can be imported, but importing coolpkg.baz.bazmod fails Suspected reason of failure: pip creates *-nspkg.path files which prevent coolpkg.baz from being found (breaking PEP420)
Test 3)DO NOT delcare namespace_packages=['coolpkg'] in setup.py of each project
Result: all modules can be imported To put it bluntly:The setup of test 2) is the one I would expect to work when writing a setup.py as described by the documentation, but pip unneccessarily creates *-nspkg.path files which undo the whole point of PEP420.
The setup of test 3) seems like an ulgy hack as we fool pip into not creating *-nspkg.path files by hiding the existance of the namepsace package from setuptools.
To summarize my questions stated above:Which is a bug, which is intended and is there a way to handle PEP420-compliant packages properly with the current versions of setuptools and pip.
Kind regards, Christoph SchmittAttachment: Example with scripts, modiy setup.py files according to test 2) and 3)
Am 17.04.2015 um 09:31 schrieb Christoph Schmitt:
I am using the newest versions of setuptools (15.0) and pip (6.1.1) with Python 3.4. I wanted to satisfy the following two requirements at the same time, but had some trouble: A) creating and installing two source distribution tarballs which have the same top level namespace package (no __init__.py for namespace package) with setuptools and pip B) having other packages within the same namespace package outside of <python>/site-packages in the PYTHONPATH (not managed by pip/setuptools)Since Python >= 3.3 supports namespace packages out of the box (PEP420) this seemed to me like a straightforward task. However, it turned out to be not. Either, setuptools would not find my modules, pip would not install them properly or requirement B) was broken. The solution that worked for me, was to omit the namespace_packages=['whatever_namespace'] declaration in setup.py (to prevent the creation of *-nspkg.pth files by pip) and not to use find_packages, which does not comply with PEP420 (no packages found).This solution is somewhat counter-intuitive and I am not sure, whether it is an intended/valid configuration of a setup.py file. Definitely, it is not clear from the documentation (which has a section about namespace packages btw.). Since I read into https://bitbucket.org/pypa/setuptools/issue/97 [1] I now know that convenient support PEP420 is not easy to achieve for setuptools (regarding find_packages). However it would have been very helpful, if the documentation explained how to handle PEP420-compliant namespace packages (without any __init__.py) in setup.py. At least I would have expected a hint that there are caveats regarding PEP420 with a link to issue 97.I also created a minimal example to reproduce the issue, which I can provide if anyone is interested.Kind regards, Christoph Schmitt _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig [2]
Links: ------ [1] https://bitbucket.org/pypa/setuptools/issue/97 [2] https://mail.python.org/mailman/listinfo/distutils-sig [3] http://mail.python.org/pipermail/distutils-sig/ [4] https://bitbucket.org/pypa/setuptools/
coolpkg-example.tar.gz
Description: GNU Zip compressed data
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
