resolving module name conflicts.
Hi, I got some code. - This code contains a package named tests - there are at least 100 references in different python files importing from above mentioned tests package. - the code also imports pytz at one place I get following warning message: /usr/lib/python2.6/dist-packages/pytz/__init__.py:32: UserWarning: Module tests was already imported from /home/user/myproject/tests/__init__.pyc, but /usr/lib/python2.6/dist-packages is being added to sys.path from pkg_resources import resource_stream Is there any way to tell pytz to import it's own tests package and tell the rest of the code to import the other? Python version is 2.6.5 Thanks in advance for any suggestion. -- http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts.
On 11/11/2011 12:27 PM Gelonida N said... Hi, I got some code. - This code contains a package named tests - there are at least 100 references in different python files importing from above mentioned tests package. - the code also imports pytz at one place I get following warning message: /usr/lib/python2.6/dist-packages/pytz/__init__.py:32: UserWarning: Module tests was already imported from /home/user/myproject/tests/__init__.pyc, but /usr/lib/python2.6/dist-packages is being added to sys.path from pkg_resources import resource_stream Is there any way to tell pytz to import it's own tests package and tell the rest of the code to import the other? Python version is 2.6.5 Thanks in advance for any suggestion. Start with http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts.
On Fri, Nov 11, 2011 at 1:27 PM, Gelonida N gelon...@gmail.com wrote:
Hi,
I got some code.
- This code contains a package named tests
- there are at least 100 references in different python files
importing from above mentioned tests package.
- the code also imports pytz at one place
I get following warning message:
/usr/lib/python2.6/dist-packages/pytz/__init__.py:32: UserWarning:
Module tests was already imported from
/home/user/myproject/tests/__init__.pyc, but
/usr/lib/python2.6/dist-packages is being added to sys.path
from pkg_resources import resource_stream
Is there any way to tell pytz to import it's own tests package and tell
the rest of the code to import the other?
This sounds like a problem caused by using relative imports.
Each import statement is made relative to some top-level module. The
import mechanism looks for it in the directories in sys.path (first
match wins). The interpreter adds the empty string to the beginning
of sys.path, by default. The empty string represents the current
working directory. An import referencing a module found there is
called a relative import.
Solution:
If possible, make sure each of your import statements is absolute (the
top-level module is in one of the [legit] sys.path directories). Then
do one of the following:
* sys.path.remove('');
* call your script from a directory without any python modules in it;
* call os.chdir(...) in your script to change CWD before making any imports.
This also means you shouldn't test your modules by just running them
as scripts. Instead, write a separate test script that imports each
of your modules absolutely (and maybe actually test them too wink).
As of 2.5, Python's from...import syntax supports explicit relative
imports, with absolute imports as the default (sort of). In 2.5 and
2.6 the feature is optional, so you must add from __future__ import
absolute_import to enable it. See PEP 328[1].
The relative import syntax means that you don't have to hard-code the
name of the packages in your imports, which helps with brevity and
portability.
The problem is that the empty string is still added to the from of
sys.path. I'm going to have to find out more about that one.
Hope that helps.
-eric
[1] http://www.python.org/dev/peps/pep-0328/
Python version is 2.6.5
Thanks in advance for any suggestion.
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts.
On Fri, Nov 11, 2011 at 2:34 PM, Eric Snow ericsnowcurren...@gmail.com wrote: The problem is that the empty string is still added to the from of sys.path. I'm going to have to find out more about that one. Okay, don't know how I missed it but the docs for sys.path[1] spell it out: As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. So if you run a module as a script, that empty string will be added to sys.path and all imports will first check the directory you were in when you ran Python... -eric [1] http://docs.python.org/library/sys.html#sys.path -- http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts.
On 11/11/2011 10:31 PM, Emile van Sebille wrote: On 11/11/2011 12:27 PM Gelonida N said... Is there any way to tell pytz to import it's own tests package and tell the rest of the code to import the other? Python version is 2.6.5 Thanks in advance for any suggestion. Start with http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports Thanks Emile / Thanks Eric The from __future__ import absolute_import is already used in most files of this project but only used for importing some other modules. All the 'tests' modules are still imported with absolute imports. 'tests' is also a top level module of the existing project and being imported from top level scripts (scripts in the python path directory) Relative imports don't work on top level scripts as it would raise ValueError: Attempted relative import in non-pack This is a legacy project where many tools assume a certain directory structure and where I'm kind of hesitant to move files into differnt directories. What's probably the most painless fix would be - force only certain (older?) versions of pytz (which don't expose this issue) to be installed. - rename tests into something else. However this will affect loads of files. and I had to check whether there are no shell scripts or custom executables, which depend on 'tests' being called 'tests' Pytz is only imported by one module, so I wondered if there were any tricks to 'change sys.path' prior to importing pytz and to make sure, that the projects 'tests' package and pytz's 'tests' package could co-exist. What I wondered is whether there was a way to fix the issue without having to 'touch' all the 100+ import statements spread over the project. In the long term I suggest, that the project's code will be changed such that it will no more conflict with pytz's tests package. (either rename the package, transform it to a sub package, ... ???), -- http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts.
On 11/11/2011 10:51 PM, Eric Snow wrote: On Fri, Nov 11, 2011 at 2:34 PM, Eric Snow ericsnowcurren...@gmail.com wrote: So if you run a module as a script, that empty string will be added to sys.path and all imports will first check the directory you were in when you ran Python... Yes that's normal (and for the rest of the code required behaviour :-( ) The top level directory is no module, but just contains scripts requiring, that '.' is in the python path. In fact theissue can be reproduced rather easily. Everything seems to work, but I don't like having this warning. Especially as the project contains a hook, which raises an alert as soon as any module tries to send data to stderr. (reporting this as potential failure of the application) $ mkdir newdir $ cd newdir $ python -c 'import pytz ; print pytz.VERSION' 2010b $ mkdir tests $ echo print 1234 tests/__init__.py $ python -c 'import pytz ; print pytz.VERSION' 2010b $ python -c 'import tests ; import pytz ; print pytz.VERSION ; reload(tests)' 1234 /usr/lib/python2.6/dist-packages/pytz/__init__.py:32: UserWarning: Module tests was already imported from tests/__init__.pyc, but /usr/lib/python2.6/dist-packages is being added to sys.path from pkg_resources import resource_stream 2010b 1234 $ What does pytz want to tell me ? -- http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts.
On Fri, 11 Nov 2011 23:11:38 +0100, Gelonida N wrote: Pytz is only imported by one module, so I wondered if there were any tricks to 'change sys.path' prior to importing pytz sys.path is just a list of paths. You can import the sys module and manipulate it any way you like. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: resolving module name conflicts. (pytz version 2010b)
On 11/12/2011 01:42 AM, Steven D'Aprano wrote: On Fri, 11 Nov 2011 23:11:38 +0100, Gelonida N wrote: Pytz is only imported by one module, so I wondered if there were any tricks to 'change sys.path' prior to importing pytz sys.path is just a list of paths. You can import the sys module and manipulate it any way you like. Hmmm, I think I didn't express myself really well. What I meant is: Is there any successful trick, which can get rid of my warning. I don't feel comfortable with warnings, that I don't understand. Please see below example (which can easily be reproduced, if you have the right (wrong) version of pytz. $ mkdir newdir $ cd newdir $ python -c 'import pytz ; print pytz.VERSION' 2010b $ mkdir tests $ echo print 1234 tests/__init__.py $ python -c 'import pytz ; print pytz.VERSION' 2010b $ python -c 'import tests ; import pytz ; print pytz.VERSION ; reload(tests)' 1234 /usr/lib/python2.6/dist-packages/pytz/__init__.py:32: UserWarning: Module tests was already imported from tests/__init__.pyc, but /usr/lib/python2.6/dist-packages is being added to sys.path from pkg_resources import resource_stream 2010b 1234 $ What does pytz want to tell me ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
Heh, so long as it works. Sorry for the delay, I've been away for a bit ;P Hope it's all owrking out -Wes -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
[EMAIL PROTECTED] wrote: I have a java program in a package called 'cmd'. This of course conflicts with the builtin python package of the same name. The thing is, I need to be able to import from both of these packages in the same script. I can import either one first, but any future attempt to import from cmd.* will look up the first cmd that was imported, so the second package is essentially eclipsed. I've tried fiddling with sys.path and sys.packageManager.searchPath, to no avail. To answer the obvious first suggestion, no I can't rename the java package to 'Cmd' or anything like that. Any ideas? -Smurf Never used it myself, but you can try to use the builtin 'imp' module. Python Library Reference 3.21 imp -- Access the import internals This module provides an interface to the mechanisms used to implement the import statement. It defines the following constants and functions: ... -- http://mail.python.org/mailman/listinfo/python-list
Module Name Conflicts
I have a java program in a package called 'cmd'. This of course conflicts with the builtin python package of the same name. The thing is, I need to be able to import from both of these packages in the same script. I can import either one first, but any future attempt to import from cmd.* will look up the first cmd that was imported, so the second package is essentially eclipsed. I've tried fiddling with sys.path and sys.packageManager.searchPath, to no avail. To answer the obvious first suggestion, no I can't rename the java package to 'Cmd' or anything like that. Any ideas? -Smurf -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
Maybe what you're looking for is __import__()?
help(__import__)
Help on built-in function __import__ in module __builtin__:
__import__(...)
__import__(name, globals, locals, fromlist) - module
Import a module. The globals are only used to determine the
context;
they are not modified. The locals are currently unused. The
fromlist
should be a list of names to emulate ``from name import ...'', or
an
empty list to emulate ``import name''.
When importing a module from a package, note that __import__('A.B',
...)
returns package A when fromlist is empty, but its submodule B when
fromlist is not empty.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
On 18 Aug 2005 16:06:46 -0700, [EMAIL PROTECTED] wrote: I have a java program in a package called 'cmd'. This of course conflicts with the builtin python package of the same name. The thing is, I need to be able to import from both of these packages in the same script. I can import either one first, but any future attempt to import from cmd.* will look up the first cmd that was imported, so the second package is essentially eclipsed. I've tried fiddling with sys.path and sys.packageManager.searchPath, to no avail. To answer the obvious first suggestion, no I can't rename the java package to 'Cmd' or anything like that. Any ideas? Assuming you can fiddle with sys.path at the right times, you can call an imported module anything you want: fix_sys_path_to_find_java_cmd_first() import cmd as java_cmd fix_sys_path_to_find_python_cmd_first() import cmd as python_cmd Obviously, then, 'cmd' does not reference either module; you'd have to use java_cmd and python_cmd as appropriate. HTH, Dan -- Dan Sommers http://www.tombstonezero.net/dan/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
Dan Sommers wrote: Assuming you can fiddle with sys.path at the right times, you can call an imported module anything you want: fix_sys_path_to_find_java_cmd_first() import cmd as java_cmd fix_sys_path_to_find_python_cmd_first() import cmd as python_cmd Obviously, then, 'cmd' does not reference either module; you'd have to use java_cmd and python_cmd as appropriate. That doesn't work. The first module is recorded as 'cmd' in sys.modules and gets reused on the second import. [~]$ mkdir foo1 [~]$ mkdir foo2 [~]$ touch foo1/blah.py [~]$ touch foo2/blah.py [~]$ python Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type help, copyright, credits or license for more information. import sys sys.path.insert(0,'foo1') import blah as blah1 sys.path.insert(0,'foo2') import blah as blah2 sys.modules['blah'] module 'blah' from 'foo1/blah.py' blah2.__file__ 'foo1/blah.py' -- Robert Kern [EMAIL PROTECTED] In the fields of hell where the grass grows high Are the graves of dreams allowed to die. -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
Robert Kern wrote: Dan Sommers wrote: Assuming you can fiddle with sys.path at the right times, you can call an imported module anything you want: fix_sys_path_to_find_java_cmd_first() import cmd as java_cmd fix_sys_path_to_find_python_cmd_first() import cmd as python_cmd Obviously, then, 'cmd' does not reference either module; you'd have to use java_cmd and python_cmd as appropriate. That doesn't work. The first module is recorded as 'cmd' in sys.modules and gets reused on the second import. Exactly. And clearing sys.modules doesn't fix the problem. Once it's imported something from the first cmd package, it can no longer find anything in another cmd package; it will always look for it in that first package. -Smurf -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
[EMAIL PROTECTED] wrote: I have a java program in a package called 'cmd'. This of course conflicts with the builtin python package of the same name. The thing is, I need to be able to import from both of these packages in the same script. I can import either one first, but any future attempt to import from cmd.* will look up the first cmd that was imported, so the second package is essentially eclipsed. I've tried fiddling with sys.path and sys.packageManager.searchPath, to no avail. To answer the obvious first suggestion, no I can't rename the java package to 'Cmd' or anything like that. Any ideas? Why not copy cmd.py into your package under a different name? -- Robert Kern [EMAIL PROTECTED] In the fields of hell where the grass grows high Are the graves of dreams allowed to die. -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
ncf wrote:
Maybe what you're looking for is __import__()?
help(__import__)
Help on built-in function __import__ in module __builtin__:
__import__(...)
__import__(name, globals, locals, fromlist) - module
Import a module. The globals are only used to determine the
context;
they are not modified. The locals are currently unused. The
fromlist
should be a list of names to emulate ``from name import ...'', or
an
empty list to emulate ``import name''.
When importing a module from a package, note that __import__('A.B',
...)
returns package A when fromlist is empty, but its submodule B when
fromlist is not empty.
Using this doesn't appear to work any better than regular old import.
-Smurf
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
Robert Kern wrote: Why not copy cmd.py into your package under a different name? It offends my sense of modularity. For the record, I'm trying to use pdb, the debugger, which in turn uses cmd. So it would be a matter of taking pdb.py and hacking it to import a renamed version of cmd... kind of messy and not a very good longterm solution. That's not to say I won't resort to it if no better options are forthcoming. -Smurf -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
[EMAIL PROTECTED] wrote: Robert Kern wrote: That doesn't work. The first module is recorded as 'cmd' in sys.modules and gets reused on the second import. Exactly. And clearing sys.modules doesn't fix the problem. Once it's imported something from the first cmd package, it can no longer find anything in another cmd package; it will always look for it in that first package. That part isn't correct. Removing the entry from sys.modules should (and has, for me, in the past) worked fine to let a second import reload a module, or find a new module after sys.path has been tweaked. Try it again. -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
I'm honestly not too sure how __import__ works, but I know you can provide a full path to it. Oh well, that was my best guess. I wish I could've been of more help. -Wes -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
ncf wrote:
Maybe what you're looking for is __import__()?
Okay, actually this does work, but only in one direction. That is, I
can import the python package first, and then the java package, but not
the other way around.
--
Importing the python cmd first:
$~ jython
Jython 2.2a1 on java1.4.2_08 (JIT: null)
Type copyright, credits or license for more information.
__import__('cmd')
module 'cmd' from '~/code/jython-2.2a1/Lib/cmd.py'
python package
import sys
sys.path.insert(0, java_classpath)
sys.modules.clear()
__import__('cmd')
java package cmd 1
java package
--
Importing the java cmd first:
$~ jython
Jython 2.2a1 on java1.4.2_08 (JIT: null)
Type copyright, credits or license for more information.
import sys
sys.path.insert(0, java_classpath)
__import__('cmd')
java package cmd 1
java package
sys.path = sys.path[1:]
sys.modules.clear()
__import__('cmd')
java package cmd 1
java package, again
--
Very odd.
-Smurf
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
On Thu, 18 Aug 2005 16:46:42 -0700, Robert Kern [EMAIL PROTECTED] wrote: Dan Sommers wrote: [ something that obviously doesn't work ] That doesn't work. The first module is recorded as 'cmd' in sys.modules and gets reused on the second import. Yes, you're right. I apologize. Regards, Dan -- Dan Sommers http://www.tombstonezero.net/dan/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
[EMAIL PROTECTED] wrote: Robert Kern wrote: Why not copy cmd.py into your package under a different name? It offends my sense of modularity. For the record, I'm trying to use pdb, the debugger, which in turn uses cmd. So it would be a matter of taking pdb.py and hacking it to import a renamed version of cmd... kind of messy and not a very good longterm solution. That's not to say I won't resort to it if no better options are forthcoming. A solution more kind to your sensibilities might be to make an auxiliary package alongside (rather than inside) yours. No modification to source necessary. pystdlib/ __init__.py cmd.py pdb.py Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type help, copyright, credits or license for more information. import pystdlib from pystdlib import pdb pdb.cmd.__file__ 'pystdlib/cmd.py' -- Robert Kern [EMAIL PROTECTED] In the fields of hell where the grass grows high Are the graves of dreams allowed to die. -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Name Conflicts
On Thu, 18 Aug 2005 16:46:42 -0700, Robert Kern [EMAIL PROTECTED] wrote:
Dan Sommers wrote:
Assuming you can fiddle with sys.path at the right times, you can call
an imported module anything you want:
fix_sys_path_to_find_java_cmd_first()
import cmd as java_cmd
fix_sys_path_to_find_python_cmd_first()
import cmd as python_cmd
Obviously, then, 'cmd' does not reference either module; you'd have to
use java_cmd and python_cmd as appropriate.
That doesn't work. The first module is recorded as 'cmd' in sys.modules
and gets reused on the second import.
[~]$ mkdir foo1
[~]$ mkdir foo2
[~]$ touch foo1/blah.py
[~]$ touch foo2/blah.py
[~]$ python
Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type help, copyright, credits or license for more information.
import sys
sys.path.insert(0,'foo1')
import blah as blah1
sys.path.insert(0,'foo2')
import blah as blah2
sys.modules['blah']
module 'blah' from 'foo1/blah.py'
blah2.__file__
'foo1/blah.py'
How about (untested)
import new
blah1 = new.module('blah')
execfile('./foo1/blah.py', blah1.__dict__)
blah2 = new.module('blah')
execfile('./foo2/blah.py', blah2.__dict__)
Of course, there is the issue of caching .pyc's and what to put in
sys.path and sys.modules, but blah1 and blah2 ought to be usable, I think.
Regards,
Bengt Richter
--
http://mail.python.org/mailman/listinfo/python-list
