[issue29144] Implicit namespace packages in Python 3.6

[Lumír Balhar]

Lumír Balhar added the comment:

Thank you for the replies. I created an issue [0] on setuptools GitHub (without 
reply yet) because I also think that setuptools should install 
pkg_resources-style __init__.py files.

[0] https://github.com/pypa/setuptools/issues/1097

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Nick Coghlan]

Nick Coghlan added the comment:

At the Python level, the rules are simple: the first directory on sys.path that 
contains an __init__.py file is identified as a self-contained package. It is 
then up to that __init__.py file to emulate namespace package behaviour (by 
extending __path__) if that's what the author intended.

Nothing changed in Python 3.6 in terms of that, and it's been that way ever 
since native namespace packages were introduced.

So if there's a behavioural change in the pkg_resources namespace emulation in 
going from Python 3.5 to 3.6 that occurs with both old & new versions of 
setuptools, then I see two main possible candidates for that:

1. Something changed in one of the APIs that setuptools uses to recalculate 
__path__

2. Something changed in importlib where we're not respecting runtime changes to 
__path__ properly, and are instead relying on either 
__spec__.submodule_search_locations or a stale cached version of __path__

Neither of those is something we *intended* to change in 3.6, so I think it's 
reasonable to categorise this as 3.6 regression at the standard library level 
(even though setuptools will likely need to work around it anyway, given the 
earliest we'll be able to fix it is in 3.6.3)

--
keywords: +3.6regression
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Brett Cannon]

Brett Cannon added the comment:

Without looking into what changed, I say Python 3.6 being more strict in what 
is a namespace package is reasonable and setuptools needs to be better about 
installing pkg_resources-using __init__.py files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Lumír Balhar]

Lumír Balhar added the comment:

Hello.

I've tried the build of python-moksha-common on Fedora rawhide [0] and Fedora 
25 [1].

Fedora rawhide: Python 2.7.13 and 3.6.1, setuptools 36.2.0
Fedora 25: Python 2.7.13 and 3.5.3, setuptools 25.1.1

Source tarball of moksha.common contains pkg_resources-style __init__.py file 
and the setuptools skipped installation of this file in all (four) cases.

It seems that setuptools do the same in all versions but the problem comes with 
Python 3.6 which is more strict than Python 3.5 in behavior related to 
namespace packages.

So, when one part of namespace package doesn't contain __init__.py file and 
other part contains pkg_resources-style __init__.py file:
- in Python 2.7 and 3.5 everything works
- in Python 3.6 submodules cannot import each other

The question is whether it is a bug/feature in Python 3.6 and setuptools should 
install pkg_resources-style __init__.py or not.

What do you think?

[0] https://koji.fedoraproject.org/koji/taskinfo?taskID=20606746
[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=20606838

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Nick Coghlan]

Nick Coghlan added the comment:

Are you able to confirm the version of setuptools involved is the same in both 
cases?

The reason I ask is that the interaction between pkg_resources-style namespace 
packages and native ones is known to be temperamental 
(https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package),
 and the suggested resolution is to make sure that if you're using 
pkg_resources namespaces at all then *all* packages contributing to the 
namespace should be using them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +brett.cannon, eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Lumír Balhar]

Lumír Balhar added the comment:

Hello.

I've met this issue again in moksha project [0] where namespace package is used.

- module moksha.common is installed via RPM to site-packages and there isn't 
file moksha/__init__.py so the implicit way to create namespace package is used 
there.
- moksha.hub module is built in a buildroot and there is moksha/__init__.py 
file so this module uses different way which causes that import of 
moksha.common is failing.

We can delete moksha/__init__.py from buildroot of moksha.hub but then we 
cannot run tests with `python setup.py test`.

Error message is:
error in moksha.hub setup command: Distribution contains no modules or packages 
for namespace package 'moksha'

The second possibility is ship moksha/__init__.py file in python-moksha-common 
RPM. The question is: Why there isn't moksha/__init__.py file in RPM even when 
the source tarball contains it? Because of setuptools!

Setuptools skips installation of __init__.py file when namespace package is 
recognized with the message:
Skipping installation of 
/builddir/build/BUILDROOT/python-moksha-common-1.2.5-1.fc27.x86_64/usr/lib/python3.6/site-packages/moksha/__init__.py
 (namespace package)

Questions summary:
- Why there is such big difference between Python 3.5 and Python 3.6?
- Why does setuptools skip installation of __init__.py file but it needs them 
to run tests of namespace subpackage?

Thank you!
Lumír

[0] https://github.com/mokshaproject/moksha

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Charalampos Stratakis]

Charalampos Stratakis added the comment:

Bumped upon a similar issue today where a package I was working on couldn't 
import a module from one of its dependencies (which was not the case in python 
3.5).

One of the lines that fail is this [0] with:
ModuleNotFoundError: No module named 'repoze.who._compat'

And the project that it depends on is this [1]

The projects are old, however what might have caused this behavior change? It 
seems kinda isolated but that the exact cause has not been pinpointed.

[0] 
https://github.com/repoze/repoze.who-sqlalchemy/blob/master/tests/databasesetup_elixir.py#L25
[1] https://github.com/repoze/repoze.who/tree/master/repoze/who

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Tomas Orsava]

Changes by Tomas Orsava :


--
nosy: +torsava

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Eric V. Smith]

Changes by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29144] Implicit namespace packages in Python 3.6

[Lumír Balhar]

New submission from Lumír Balhar:

Hello.

I've found a really strange difference between Python 3.5 and 3.6 related to 
namespace packages and I cannot find any note in changelogs.

I've created a simple bash script which can reproduce my issue using virtual 
environments: http://pastebin.com/j3fXMtR4

This script creates a virtual environment, installs marrow.util from PyPI which 
is one part of namespace package and creates the other part (marrow.mailer) in 
the local folder. Then script tries to import marrow.util and marrow.mailer - 
and there is the difference.

In Python 3.5 - I can import marrow.util (installed via pip) but I cannot 
import local marrow.mailer module. I think that this is right behavior.
In Python 3.6 - I cannot import marrow.util (installed via pip) but I can 
import local marrow.mailer module.

Python versions are 3.5.2 (installed from Fedora repositories) and 3.6.0 
(compiled manually).

Why there is such a big difference? Am I missing something?

I tried to ask on IRC, stackoverflow etc. but without success. Could you please 
explain the reason for this behavior to me?

Thank you and have a nice day.
Lumír

Bash script attached

--
files: test_namespace_package.sh
messages: 284587
nosy: frenzy
priority: normal
severity: normal
status: open
title: Implicit namespace packages in Python 3.6
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46129/test_namespace_package.sh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com