Re: How to fix my imports/file structure
On 1/20/2016 8:26 PM, Travis Griggs wrote: I wrote a simple set of python3 files for emulating a small set of mongodb features on a 32 bit platform. I fired up PyCharm and put together a directory that looked like: minu/ client.py database.py collection.py test_client.py test_database.py test_client.py My imports are simple. For example, client.py has the following at the top: from collection import Collection Basically, client has a Client class, collection has a Collection class, and database has a Database class. Not too tough. As long as I cd into the minu directory, I can fire up a python3 interpreter and do things like: >>> from client import Client >>> c = Client(pathstring='something’) And everything just works. I can run the test_files as well, which use the same sorts of imports. I'd like to modularize this, so I can use it another project by just dropping the minu directory alongside my application's .py files and just have everything work. E.g. SomeDirectory/ application.py minu/ … and application.py does something like: from minu.client import Client When I try this though, and am running python3 from another directory, the local imports don't work. I placed an empty init.py in the minu directory. That made it so I could import minu. But the others broke. I tried using things like from .collection import Collection #added the dot but then I can't run things in the original directory anymore, like I could before. What is the simple/right way to do this? I have looked around a bit with Dr. Google, but none of the examples really clarify this well (at least, for me), feel free to point out the one I missed. Summary: there are two solutions, one of which I think is better. Imports of python-coded files are done by searching the directories, usually not packages, on sys.path. Python prepends '' to sys.path, standing for the 'directory that contain the startup file'. In the first case, startup is client.py and '' is 'minu', so other modules in minu can be imported directly. In the second case, startup is application.py and '' is SomeDirectory, so other modules in SomeDirectory can be imported. There is only one other module, minu. The modules within minu must be imported via Minu. What you want to do is make the two situations identical with respect to minu module imports, so the import code will work the same in the two scenarios. 1. Make scenario 2 match scenario 1. In application.py, add sys.path.insert(1,path/to/minu). Then import minu modules directly. 2. Make scenario 1 match scenario 2 in the sense of having the directory containing minu be on sys.path. But be clever. Suppose minu lives in directory Projects. In your 3.x Lib/site-packages directory, add file 'Projects.pth' containing path/to/minu (without quotes!). This makes Projects an extension of site-packages, and inserts minu 'virtually' into site-packages, so indirect imports for any user. The negative is having to do this for every 3.x version you want to use minu with. But new versions only appear every 18-24 months, and copying the file into a new site-packages is not a big deal. The positive is that you no longer need to drop minu into SomeDirectory but can import its modules into any application module as if minu were in site-packages, and as it would (should) be if you were to distribute minu to others. That means that the application is always using the latest version of minu without re-dropping. This also means that you can import minu and its module into interactive Python or IDLE's shell, without touching sys.path. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
How to fix my imports/file structure
I wrote a simple set of python3 files for emulating a small set of mongodb features on a 32 bit platform. I fired up PyCharm and put together a directory that looked like: minu/ client.py database.py collection.py test_client.py test_database.py test_client.py My imports are simple. For example, client.py has the following at the top: from collection import Collection Basically, client has a Client class, collection has a Collection class, and database has a Database class. Not too tough. As long as I cd into the minu directory, I can fire up a python3 interpreter and do things like: >>> from client import Client >>> c = Client(pathstring='something’) And everything just works. I can run the test_files as well, which use the same sorts of imports. I'd like to modularize this, so I can use it another project by just dropping the minu directory alongside my application's .py files and just have everything work. E.g. SomeDirectory/ application.py minu/ … and application.py does something like: from minu.client import Client When I try this though, and am running python3 from another directory, the local imports don't work. I placed an empty init.py in the minu directory. That made it so I could import minu. But the others broke. I tried using things like from .collection import Collection #added the dot but then I can't run things in the original directory anymore, like I could before. What is the simple/right way to do this? I have looked around a bit with Dr. Google, but none of the examples really clarify this well (at least, for me), feel free to point out the one I missed. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to fix my imports/file structure
On Thursday 21 January 2016 12:26, Travis Griggs wrote:
> I wrote a simple set of python3 files for emulating a small set of mongodb
> features on a 32 bit platform. I fired up PyCharm and put together a
> directory that looked like:
>
> minu/
> client.py
> database.py
> collection.py
> test_client.py
> test_database.py
> test_client.py
This will only work so long as you cd into the minu directory first.
To fix that, you can either:
(1) Add minu to your path. You can put this at the start of your script:
import sys
if "/path/to/minu" not in sys.path:
sys.path.append("/path/to/minu")
(where "/path/to/minu" is the absolute path to the actual directory).
You need this to occur before you start importing from minu.
Or you can adjust the path using an environment variable:
export PYTHONPATH="/path/to/minu"
before you launch Python. (If you're using Linux or Unix, you could put that
in your .bashrc, or equivalent.)
Or you can create a .pth file that points to your minu directory.
(I haven't tried this, I might have some of the details wrong.)
In your Python site-packages directory (usually found somewhere like
/usr/local/lib/python3.4/site-packages/) drop a file called "minu.pth"
containing a single line of text:
/absolute/path/to/minu
then that path will be automatically added to your python path and you'll be
able to import any .py file in minu from anywhere.
https://docs.python.org/3/library/site.html
(2) Alternatively, turn minu into a package, rather than a directory of
unrelated modules.
- Create a file __init__.py and put it in the minu directory.
- Place the minu directory somewhere in your Python path. Or use a .pth
file, as above.
Now minu is a *package*, and the modules "client.py", "collection.py" etc.
are assumed to collaborate rather than be a random collection of arbitrary
modules. From *outside* of minu, you can import submodules of the package:
import minu.collection
from minu.client import Spam
and from submodules *inside* minu, you can either use the same absolute
imports as above, or you can use relative imports:
# inside minu.client
from .collection import Collection
Google for "relative and absolute imports" for more info.
> My imports are simple. For example, client.py has the following at the
> top:
>
> from collection import Collection
>
> Basically, client has a Client class, collection has a Collection class,
> and database has a Database class. Not too tough.
Python isn't Java, you don't have to force each class to live in its own
file. Sounds like you might simplify the whole job by putting everything in
a single minu.py file.
> As long as I cd into the minu directory, I can fire up a python3
> interpreter and do things like:
>
> >>> from client import Client
> >>> c = Client(pathstring='something’)
>
> And everything just works. I can run the test_files as well, which use the
> same sorts of imports.
>
> I'd like to modularize this, so I can use it another project by just
> dropping the minu directory alongside my application's .py files and just
> have everything work. E.g.
>
> SomeDirectory/
> application.py
> minu/
> …
>
> and application.py does something like:
>
> from minu.client import Client
Sounds like you want minu to be a package. See (2) above.
> When I try this though, and am running python3 from another directory, the
> local imports don't work. I placed an empty init.py in the minu directory.
> That made it so I could import minu. But the others broke. I tried using
> things like
You still need minu to be somewhere that the main script can see, that is,
in one of the locations listed in sys.path, but that's not hard to set up.
If you run:
python3 SomeDirectory/application.py
SomeDirectory will be automatically added to the path and in theory it
should all Just Work.
If not, start by printing sys.path and see what you have.
> from .collection import Collection #added the dot
>
> but then I can't run things in the original directory anymore, like I
> could before. What is the simple/right way to do this?
I would say that the right solution is to modify the imports in the minu
submodules to use package absolute and/or relative import syntax. In theory
you can probably get it to work by adding sufficient directories to the
path, but that gets ugly and messy quickly.
Packages were invented to solve your problem, but the submodules do have to
be written as if they are inside a package, and not as stand alone modules.
--
Steve
--
https://mail.python.org/mailman/listinfo/python-list
