Re: [Python-Dev] Relative imports in Py3k
On Mon, Oct 11, 2010 at 1:54 AM, anatoly techtonik techto...@gmail.com wrote: On Sun, Sep 26, 2010 at 2:32 PM, Nick Coghlan ncogh...@gmail.com wrote: This is almost certainly failing because the directory containing the spyderlib package isn't on sys.path anywhere (instead, whichever directory contains the script you executed directly will be in there, which will be somewhere inside the package instead of outside it). Put the appropriate directory in PYTHONPATH and these tests should start working. This is a hack. I use relative imports, because I don't want to care about PYTHONPATH issues. I work with two clones of spyderlib (reference point and feature branch). You propose to switch PYTHONPATH every time I want to execute debug code in the __main__ section from either of them. Anatoly, unconstructive responses like this are why people often react negatively to your attempts to be helpful. I specifically mentioned 2 things you could do: - modify PYTHONPATH - use -m to execute your modules and just switch your current working directory depending on which version of spyderlib you want to execute Responding to that by telling me that modifying PYTHONPATH when working with in-development code is clumsy isn't telling me anything I don't already know. There's a *reason* why you can now use -m to have relative imports just work so long as your current working directory includes the top level directory for your package. There's a *reason* we added the ability to execute directories and zipfiles and have them automatically add themselves to the head of sys.path. And it's the same reason: we've been working on making it easier to work with uninstalled (i.e. not on PYTHONPATH) code for years. Throwing your hands up and saying bah, it doesn't work, it should just read my mind and know what I meant! really isn't helpful. Based on your latest response, giving your spyderlib package a top-level __main__.py and running the directory directly may even be an option for you. But when your response solely discusses PYTHONPATH without even mentioning the better alternative I offered (i.e. using -m and just switching your current working directory in a command shell), it's hard to assess your actual use case. As far as documentation goes, I personally find it incredibly difficult to write simple, user-oriented documentation of the import system, as I've long since lost my perspective on what's simple and what's complicated when it comes to the import system. I'd love for someone to tackle the task of writing clear end-user documentation of the whole PEP 302, 328, 338, 366, imp, runpy, importlib, etc arrangement (more comprehensive documentation of the zipfile and directory execution from issue 1739468 wouldn't hurt either). The problem is that the people who already know enough to write such documentation don't need it, and anyone else that sets out to learn enough to be able to write it discovers that there are so many backwards compatibility hacks that complicate an otherwise reasonably clean design that they throw their hands up in despair. (That said, I did attempt to write such a piece of documentation a few years back [1] so anyone that wanted to try it could at least consider using that as a starting point instead of starting with a blank screen) Cheers, Nick. [1] http://svn.python.org/view/sandbox/trunk/userref/ODF/Chapter07_ModulesAndApplications.odt -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On 10/11/2010 07:27 AM, Nick Coghlan wrote: On Mon, Oct 11, 2010 at 1:54 AM, anatoly techtoniktechto...@gmail.com wrote: On Sun, Sep 26, 2010 at 2:32 PM, Nick Coghlanncogh...@gmail.com wrote: This is almost certainly failing because the directory containing the spyderlib package isn't on sys.path anywhere (instead, whichever directory contains the script you executed directly will be in there, which will be somewhere inside the package instead of outside it). Put the appropriate directory in PYTHONPATH and these tests should start working. This is a hack. I use relative imports, because I don't want to care about PYTHONPATH issues. I work with two clones of spyderlib (reference point and feature branch). You propose to switch PYTHONPATH every time I want to execute debug code in the __main__ section from either of them. Anatoly, unconstructive responses like this are why people often react negatively to your attempts to be helpful. I specifically mentioned 2 things you could do: - modify PYTHONPATH - use -m to execute your modules and just switch your current working directory depending on which version of spyderlib you want to execute I don't recall Anatoly saying which p3k version and revision he was using. Relative imports was broken for while in 3.2. It's fixed now and I presume he is using a fairly current revision of 3.2. When you do a make install for 3.2 on Ubuntu, the current directory path , isn't perpended to sys.path. I don't know if that is an over site or not, but it could be a factor. A few more suggestions ... Make A test runner script which modifies sys.path. It also could be considered a hack, but it doesn't require modifying PYTHONPATH, so it wouldn't have any potential to have side effects on other modules/programs. One of my personal choices when writing large applications (rather than scripts), is to make a local lib directory and prepend that to sys.path in the main application file before any local imports. # Add a local lib to the search path. lib = os.path.abspath(os.path.join(__file__, '..', 'lib')) sys.path.insert(0, lib) [Appliction dir not in PYthon path] main_app_file.py test.py [lib] [test package] ... #test modules ... #other local modules and packages I then add a -test option to the main_app_file.py or a create test.py file at the same level as the main_app_file. The test runner also needs to add lib to sys.path, but after that it can import and find any/all tests you want to run. The test modules can use relative imports as long as they aren't circular. * The error message in the case of circular imports could be much better! Cheers, Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On Sat, Sep 25, 2010 at 4:52 PM, Georg Brandl g.bra...@gmx.net wrote: I wonder if situation with relative imports in packages is improved in Python 3k or we are still doomed to a chain of hacks? My user story: ... PEP 328 http://www.python.org/dev/peps/pep-0328/ proposes: from ... import config from ..utils.qthelpers import translate, add_actions, create_action But this doesn't work, and I couldn't find any short user level explanation why it is not possible to make this work at least in Py3k without additional magic. Uh, this doesn't work is a report that every developer loves. I suppose Python does raise an exception with a message? It is not a bug report. It is a user story - that means the thing that I, as a user of Python, assumed to work. But instead have got: ValueError: Attempted relative import in non-package There are __init__.py files in every directory of spyderlib, but I don't want to debug or seek a workarounds against this in Python 2. I just want to know if relative imports are fixed in Python 3. -- anatoly t. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On Sun, Sep 26, 2010 at 2:32 PM, Nick Coghlan ncogh...@gmail.com wrote: I wonder if situation with relative imports in packages is improved in Python 3k or we are still doomed to a chain of hacks? This is The question. Is Python 3k more friendly to users or require them to learn the zen of import (which is not available in concise format, unfortunately, unlike this thing)? relatives. All imports are like: from spyderlib.config import get_icon from spyderlib.utils.qthelpers import translate, add_actions, create_action This is almost certainly failing because the directory containing the spyderlib package isn't on sys.path anywhere (instead, whichever directory contains the script you executed directly will be in there, which will be somewhere inside the package instead of outside it). Put the appropriate directory in PYTHONPATH and these tests should start working. This is a hack. I use relative imports, because I don't want to care about PYTHONPATH issues. I work with two clones of spyderlib (reference point and feature branch). You propose to switch PYTHONPATH every time I want to execute debug code in the __main__ section from either of them. Of course, I can write scripts to automate the thing, but it is a _basic_ debug operation, and as a one of the hundreds Python users I don't want to go through this misery just to be able to run python submodule.py. Hope this clarifies The question. PEP 328 http://www.python.org/dev/peps/pep-0328/ proposes: from ... import config from ..utils.qthelpers import translate, add_actions, create_action This fails for two reasons: 1. __main__ is missing the module namespace information PEP 328 needs to do its thing 2. Even if 1 is resolved, PEP 328 will resolve to the same absolute imports you're already using and likely fail for the same reason (i.e. spyderlib not found on sys.path) This is a hack. There is no explanation why this hack is required, or it is not a user-friendly explanation. But this doesn't work, and I couldn't find any short user level explanation why it is not possible to make this work at least in Py3k without additional magic. If you use the -m switch to run your module from the directory that contains the spyderlib package directory, it will work. The use of -m provides the module namespace information that PEP 328 needs, while running from the directory that contains the spyderlib package ensures it is visible through sys.path. The one caveat is that the specified module is run as __main__ and hence does not exist in sys.modules under its normal name. If it gets imported by another module, then you will end up with two copies of the module (one as __main__ and one under the normal name). This may or may not cause problems, depending on the nature of the module being executed. While PEP 366 describes the boilerplate you can put at the top of your module to allow a directly executed module to try to find its siblings, it still requires that PYTHONPATH be set appropriately. And if you set PYTHONPATH appropriately, then direct execution with absolute imports will work. (The explanation of the failures applies for all Python versions that I am aware of, but the -m based fix only became available in 2.6) (The impact of various command line options and the PYTHONPATH environment variable on sys.path are described at http://docs.python.org/using/cmdline.html) (The basic import search algorithm is described in the tutorial: http://docs.python.org/tutorial/modules.html#the-module-search-path) (And the full gory specification details are in PEP 302, with a few subsequent tweaks courtesy of PEP 328 and PEP 366). This what I call zen of import. I don't see why such awkward way should be necessary in Python 3k, which breaks backwards compatibility. Why it can't just work for my user story? -- anatoly t. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On Sun, 10 Oct 2010 18:54:20 +0300, anatoly techtonik techto...@gmail.com wrote: (The explanation of the failures applies for all Python versions that I am aware of, but the -m based fix only became available in 2.6) (The impact of various command line options and the PYTHONPATH environment variable on sys.path are described at http://docs.python.org/using/cmdline.html) (The basic import search algorithm is described in the tutorial: http://docs.python.org/tutorial/modules.html#the-module-search-path) (And the full gory specification details are in PEP 302, with a few subsequent tweaks courtesy of PEP 328 and PEP 366). This what I call zen of import. I don't see why such awkward way should be necessary in Python 3k, which breaks backwards compatibility. Why it can't just work for my user story? Because you weren't around advocating and implementing a change when Python 3 was developed? It's too late now to arbitrarily break backward compatibility, so you'll have to advocate and develop any change inside the parameters of Python's normal backward compatibility policy. In other words, you should move this discussion to python-ideas. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On Sun, Sep 26, 2010 at 7:46 AM, Martin v. Löwis mar...@v.loewis.de wrote: What you read as a bug report was labeled user story, which I think is anatoly's way of saying it's not a bug report. Skimming the original post, I actually thought of two possible candidates that fit the it doesn't work moniker when it comes to imports: http://bugs.python.org/issue8098 (multi-threaded dodginess courtesy of some deadlock avoidance we added a while back) http://bugs.python.org/issue992389 (ah, the joys of circular imports - our dear little 6 year old tracker issue...) Not quite what anatoly is talking about though (see other post). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On Sat, Sep 25, 2010 at 11:15 PM, anatoly techtonik techto...@gmail.com wrote: Hi, I wonder if situation with relative imports in packages is improved in Python 3k or we are still doomed to a chain of hacks? My user story: I am currently debugging project, which consists of many modules in one package. Each module has tests or other useful stuff for debug in its main section, but it is a disaster to use it, because I can't just execute the module file and expect it to find relatives. All imports are like: from spyderlib.config import get_icon from spyderlib.utils.qthelpers import translate, add_actions, create_action This is almost certainly failing because the directory containing the spyderlib package isn't on sys.path anywhere (instead, whichever directory contains the script you executed directly will be in there, which will be somewhere inside the package instead of outside it). Put the appropriate directory in PYTHONPATH and these tests should start working. PEP 328 http://www.python.org/dev/peps/pep-0328/ proposes: from ... import config from ..utils.qthelpers import translate, add_actions, create_action This fails for two reasons: 1. __main__ is missing the module namespace information PEP 328 needs to do its thing 2. Even if 1 is resolved, PEP 328 will resolve to the same absolute imports you're already using and likely fail for the same reason (i.e. spyderlib not found on sys.path) But this doesn't work, and I couldn't find any short user level explanation why it is not possible to make this work at least in Py3k without additional magic. If you use the -m switch to run your module from the directory that contains the spyderlib package directory, it will work. The use of -m provides the module namespace information that PEP 328 needs, while running from the directory that contains the spyderlib package ensures it is visible through sys.path. The one caveat is that the specified module is run as __main__ and hence does not exist in sys.modules under its normal name. If it gets imported by another module, then you will end up with two copies of the module (one as __main__ and one under the normal name). This may or may not cause problems, depending on the nature of the module being executed. While PEP 366 describes the boilerplate you can put at the top of your module to allow a directly executed module to try to find its siblings, it still requires that PYTHONPATH be set appropriately. And if you set PYTHONPATH appropriately, then direct execution with absolute imports will work. (The explanation of the failures applies for all Python versions that I am aware of, but the -m based fix only became available in 2.6) (The impact of various command line options and the PYTHONPATH environment variable on sys.path are described at http://docs.python.org/using/cmdline.html) (The basic import search algorithm is described in the tutorial: http://docs.python.org/tutorial/modules.html#the-module-search-path) (And the full gory specification details are in PEP 302, with a few subsequent tweaks courtesy of PEP 328 and PEP 366). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Relative imports in Py3k
Hi, I wonder if situation with relative imports in packages is improved in Python 3k or we are still doomed to a chain of hacks? My user story: I am currently debugging project, which consists of many modules in one package. Each module has tests or other useful stuff for debug in its main section, but it is a disaster to use it, because I can't just execute the module file and expect it to find relatives. All imports are like: from spyderlib.config import get_icon from spyderlib.utils.qthelpers import translate, add_actions, create_action PEP 328 http://www.python.org/dev/peps/pep-0328/ proposes: from ... import config from ..utils.qthelpers import translate, add_actions, create_action But this doesn't work, and I couldn't find any short user level explanation why it is not possible to make this work at least in Py3k without additional magic. -- anatoly t. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
Am 25.09.2010 15:15, schrieb anatoly techtonik: Hi, I wonder if situation with relative imports in packages is improved in Python 3k or we are still doomed to a chain of hacks? My user story: I am currently debugging project, which consists of many modules in one package. Each module has tests or other useful stuff for debug in its main section, but it is a disaster to use it, because I can't just execute the module file and expect it to find relatives. All imports are like: from spyderlib.config import get_icon from spyderlib.utils.qthelpers import translate, add_actions, create_action PEP 328 http://www.python.org/dev/peps/pep-0328/ proposes: from ... import config from ..utils.qthelpers import translate, add_actions, create_action But this doesn't work, and I couldn't find any short user level explanation why it is not possible to make this work at least in Py3k without additional magic. Uh, this doesn't work is a report that every developer loves. I suppose Python does raise an exception with a message? For diagnosing this, you should also at least include from which module you're trying to executing these import statements. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
On 9/25/2010 9:15 AM, anatoly techtonik wrote: from ... import config from ..utils.qthelpers import translate, add_actions, create_action But this doesn't work, and I couldn't find any short user level explanation why it is not possible to make this work at least in Py3k without additional magic. This doesn't belong on python-dev, as it's not about developing python. Also, it's a horrible bug report, if that's what it is. I'd suggest trying python-list and reading something like http://itscommonsensestupid.blogspot.com/2008/07/tips-to-write-good-bug-report.html Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Relative imports in Py3k
Also, it's a horrible bug report, if that's what it is. It's not a bug report, and I don't think it was meant to be one. It started with I wonder if, suggesting that it is really a request for help. What you read as a bug report was labeled user story, which I think is anatoly's way of saying it's not a bug report. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
