[issue16570] Absolute imports fail to take full path into account?
New submission from Dirkjan Ochtman: This seems confusing: djc@enrai test $ ls -lR .: total 0 drwxr-xr-x 4 djc users 160 Nov 28 15:35 pkg ./pkg: total 4 -rw-r--r-- 1 djc users 39 Nov 28 15:37 http.py -rw-r--r-- 1 djc users 0 Nov 28 15:34 __init__.py drwxr-xr-x 3 djc users 136 Nov 28 15:40 tests ./pkg/tests: total 8 -rw-r--r-- 1 djc users 21 Nov 28 15:37 http.py -rw-r--r-- 1 djc users 27 Nov 28 15:40 __init__.py djc@enrai test $ cat pkg/__init__.py djc@enrai test $ cat pkg/http.py from http.client import HTTPConnection djc@enrai test $ cat pkg/tests/__init__.py from pkg.tests import http djc@enrai test $ cat pkg/tests/http.py from pkg import http djc@enrai test $ python3 Python 3.2.3 (default, May 28 2012, 09:27:08) [GCC 4.5.3] on linux2 Type help, copyright, credits or license for more information. from pkg import http djc@enrai test $ python3 Python 3.2.3 (default, May 28 2012, 09:27:08) [GCC 4.5.3] on linux2 Type help, copyright, credits or license for more information. from pkg.tests import http djc@enrai test $ PYTHONPATH=. python3 pkg/tests/__init__.py Traceback (most recent call last): File pkg/tests/__init__.py, line 1, in module from pkg.tests import http File /home/djc/src/test/pkg/tests/__init__.py, line 1, in module from pkg.tests import http File /home/djc/src/test/pkg/tests/http.py, line 1, in module from pkg import http File /home/djc/src/test/pkg/http.py, line 1, in module from http.client import HTTPConnection File /home/djc/src/test/pkg/tests/http.py, line 1, in module from pkg import http ImportError: cannot import name http -- components: Interpreter Core messages: 176545 nosy: brett.cannon, djc, ncoghlan priority: normal severity: normal status: open title: Absolute imports fail to take full path into account? versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Brett Cannon added the comment: This test example is nonsensical. You are directly executing a package which leads to pkg/tests being put in sys.path first. That means you end up with http in pkg/tests masking the one in the stdlib, which causes your ``from http.clients import HTTPConnection`` to fail. You should be using the -m option with a proper __main__.py file to avoid this. If you do that with a __main__.py file doing nothing more than ``from pkg import tests`` everything works. -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Dirkjan Ochtman added the comment: It's not nonsensical. Something exactly like it came up while I was looking into porting couchdb-python to Python 3. couchdb-python has had a couchdb.http module for a while now; it didn't start clashing with the stdlib until Python 3. Also, I don't understand why Python is trying to import any bare http, when all the imports are absolute. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Dirkjan Ochtman added the comment: Never mind, I get what you're getting at now. I thought that the cwd was added to the sys.path, not the containing directory for the executed script. I'll look into using __main__. Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Brett Cannon added the comment: When you execute a module on the command line, sys.path[0] is set to that containing directory (e.g. pkg/tests in your example; just have pkg/tests/__init__.py print out sys.path to see what I mean). ``import http`` is going to look on sys.path no matter what, and with ``pkg/tests`` being the first entry on sys.path, its going to find pkg/tests/http.py before it even has a chance to look in the directory containing the stdlib. A bare import only means look on sys.path, not magically only look in the stdlib. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Brett Cannon added the comment: Glad it makes sense now. =) I was writing my reply while you sent yours. Anyway, your PYTHONPATH setting goes on to sys.path *after* the directory containing the script being executed. This is so that when you execute Python code somewhere it will get the local modules and packages it expects and not some random one you accidentally masked using PYTHONPATH. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Eric Snow added the comment: Things are working as they should here. The key points: * when running a script, sys.path[0] is set to the script's directory (with no script sys.path[0] is set to the CWD, see issue13475), * pkg/tests/__init__.py is loaded and executed twice: once as the script under the __main__ module and once during import as pkg.tests, * Python does not handle circular imports very well, * bad things happen when a top-level module has the same name as a module in the stdlib. Together those explain what's going on. The import you did at the REPL happened with '' at sys.path[0], so modules are found at the right relative places: pkg, pkg.http, pkg.tests, and pkg.tests.http. There is no script involved, just imports, so no double-loading happens. Things go south when you run PYTHONPATH=. python3 pkg/tests/__init__.py. First of all, pkg/tests/__init__.py is executed twice: once as the script and once under import. Though you set PYTHONPATH, sys.path[0] is set to pkg/tests, the directory the script is in. sys.path[1] is ., what you were expecting to be at sys.path[0]. So when finding modules, the import system will first look in pkg/tests then in .. Thus the pkg.* imports work as expected. However, from http.client import HTTPConnection in pkg/tests/http.py finds the same http.py (this time as the http module instead of pkg.tests.http) in pkg/tests rather than the stdlib module as you expected. So it tries to import it a second time with a different module name. Since pkg/tests/http.py is already being loaded due to pkg/test/__init__.py, you get a circular import. Even if you did not get the circular import you would have gotten an ImportError for http.client since pkg/tests/http.py neither behaves li ke a package nor actually has any client submodule. Part of the fix is to use relative imports where appropriate. For instance, change pkg/tests/__init__.py like this: from . import http Also, don't run pkg/tests/__init__.py directly. Instead try this: PYTHONPATH=. python3 -m pkg.tests However, this implies that you wanted to run the package as a script, so you should have pkg/tests/__main__.py which would import pkg.tests. Alternately, you could have a dedicated script elsewhere, perhaps next to the pkg directory that does the same thing. Here's what I mean: some_project/ pkg/ tests/ __init__.py __main__.py (option 1) http.py __init__.py run_unittests.py (option 2) Finally, don't name your modules with the same names as those in the stdlib. wink -- nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16570] Absolute imports fail to take full path into account?
Eric Snow added the comment: oops. too slow. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16570 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
