Re: Import module from a different subdirectory

[Peter J. Holzer]

On 2019-05-18 16:15:34 -0700, Rich Shepard wrote:
> My apologies to all who patiently tried to get me to see what I kept
> missing.

I've certainly made similar mistakes in the past (and probably will in
the future).

And I didn't see it when I read your mail the first time. But then I
read Piet's reply and thought "But Rich already wrote that ..." and
reread you message. And then I was like "Wait a minute ..."[1]

hp

[1] And now I've got Frank Zappa's "Valley Girl" stuck in my head. 
One shouldn't compose a reply while waiting for the coffee machine
to boot - too much time to form weird associations.

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Rich Shepard]

On Sun, 19 May 2019, Peter J. Holzer wrote:


This won't help much if your directory named "business-tracker" (see
above).


Peter, et al.:

Yep. User error. The directory is actually 'business_tracker' and I used the
application name, 'bustrac', instead when I set PYTHONPATH. Discovered this
a bit ago when I looked over the messages in the thread and realized that I
had a major brain cramp when setting up PYTHONPATH.

Having corrected this, the problem has gone away.

My apologies to all who patiently tried to get me to see what I kept
missing.

My thanks to all,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Peter J. Holzer]

On 2019-05-18 09:43:34 -0700, Rich Shepard wrote:
> The project layout, briefly, is:
> 
> ~/development/business-tracker/

>   classes/
>   gui/
> 
> All subdirectories contain a __init__.py file to identify them as packages.
> 'classes/' contains model.py; 'gui/' contains test_act_de.py.
> 
> The project root directory is present in both PYTHONPATH and sys.path:
> 
> $ echo $PYTHONPATH
> /home/rshepard/development/bustrac/
 ^^^

This won't help much if your directory named "business-tracker" (see
above).

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Piet van Oostrum]

Rich Shepard  writes:

>
> $ python3
> Python 3.7.3 (default, Mar 26 2019, 06:40:28) [GCC 5.5.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
 import sys
 sys.path
> ['', '/home/rshepard/development/bustrac', '/usr/lib/python37.zip',
> '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload',
> '/usr/lib/python3.7/site-packages']
>
> When the gui/ subdirectory is the current working directory and I try to run
> text_act_de.py I get the error that model.py is not found in the classes
> subdirectory. I don't see how my directory structure, file locations, and
> paths differ from your examples.

In a previous message the error was that 'classes' wasn't found, not that 
model.py wasn't found in classes, i.e. it was the same error as Peter got.

So to get this working you must make sure 'classes' is inside a directory that 
is in sys.path, for example by adding:

sys.path.insert(0, '..')
-- 
Piet van Oostrum 
WWW: http://piet.vanoostrum.org/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Rich Shepard]

On Sat, 18 May 2019, Peter J. Holzer wrote:


"" is in sys.path, so "classes" and classes.model are found.

Now lets go to a different subdirectory:
This doesn't work, since there is no classes/model.py in "", only in "..".

But if I add a PYTHONPATH, it works again:


Peter,

The project layout, briefly, is:

~/development/business-tracker/
classes/
gui/

All subdirectories contain a __init__.py file to identify them as packages.
'classes/' contains model.py; 'gui/' contains test_act_de.py.

The project root directory is present in both PYTHONPATH and sys.path:

$ echo $PYTHONPATH
/home/rshepard/development/bustrac/

$ python3
Python 3.7.3 (default, Mar 26 2019, 06:40:28) 
[GCC 5.5.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

import sys
sys.path

['', '/home/rshepard/development/bustrac', '/usr/lib/python37.zip', 
'/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', 
'/usr/lib/python3.7/site-packages']

When the gui/ subdirectory is the current working directory and I try to run
text_act_de.py I get the error that model.py is not found in the classes
subdirectory. I don't see how my directory structure, file locations, and
paths differ from your examples.

Regards,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Peter J. Holzer]

On 2019-05-18 05:45:23 -0700, Rich Shepard wrote:
> On Sat, 18 May 2019, dieter wrote:
> > > > sys.path
> ['', '/home/rshepard/development/bustrac', '/usr/lib/python37.zip',
> '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload',
> '/usr/lib/python3.7/site-packages']
> 
> All directories are present, and the one for the current project is the
> first one searched.
[...]
> I understand this. Within ~/development/bustrac as the CWD when I invoke the
> test application with the first line:
> 
> from classes import model as m
> 
> I still see this result:
> 
> $ python3 test_act_de.py Traceback (most recent call last):
>   File "test_act_de.py", line 1, in 
> from classes import model as m
> ModuleNotFoundError: No module named 'classes'
> 
> 'classes' is a package (a subdirectory) containing an empty __init__.py, not
> a module. 'classes' conains a single module, model.py.

This works for me:

hrunkner:~ 18:14 :-) 1141% cd tmp 
hrunkner:~/tmp 18:14 :-) 1142% mkdir bustrac  
hrunkner:~/tmp 18:14 :-) 1143% cd bustrac   
hrunkner:~/tmp/bustrac 18:14 :-) 1144% mkdir classes 
hrunkner:~/tmp/bustrac 18:14 :-) 1145% touch classes/__init__.py
hrunkner:~/tmp/bustrac 18:14 :-) 1146% touch classes/model.py   
hrunkner:~/tmp/bustrac 18:14 :-) 1147% python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from classes import model
>>> 

"" is in sys.path, so "classes" and classes.model are found.

Now lets go to a different subdirectory:

hrunkner:~/tmp/bustrac 18:15 :-) 1148% mkdir scripts
hrunkner:~/tmp/bustrac 18:15 :-) 1149% cd scripts 
hrunkner:~/tmp/bustrac/scripts 18:15 :-) 1150% python3  
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from classes import model
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named 'classes'
>>> 

This doesn't work, since there is no classes/model.py in "", only in "..".

But if I add a PYTHONPATH, it works again:

hrunkner:~/tmp/bustrac/scripts 18:15 :-) 1151% export 
PYTHONPATH=~/tmp/bustrac
hrunkner:~/tmp/bustrac/scripts 18:15 :-) 1152% python3  
  
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from classes import model
>>> 


-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Rich Shepard]

On Sat, 18 May 2019, dieter wrote:


Test this by looking at "sys.path" instead:

import sys
sys.path

It is "sys.path" which actually controls the import machinery.


Dieter,

Thank you. I missed this when researching PYTHONPATH. Here's what I get:


sys.path

['', '/home/rshepard/development/bustrac', '/usr/lib/python37.zip',
'/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload',
'/usr/lib/python3.7/site-packages']

All directories are present, and the one for the current project is the
first one searched.


"sys.path" is typically a sequence of folders. Python's import machinery
will look in those folders for modules/paackages for its (absolute)
imports. Thus, if you use "import " or "from  import ...", then
one of those folders should contain a module or package named "".


I understand this. Within ~/development/bustrac as the CWD when I invoke the
test application with the first line:

from classes import model as m

I still see this result:

$ python3 test_act_de.py 
Traceback (most recent call last):

  File "test_act_de.py", line 1, in 
from classes import model as m
ModuleNotFoundError: No module named 'classes'

'classes' is a package (a subdirectory) containing an empty __init__.py, not
a module. 'classes' conains a single module, model.py.

Since the project's root directory is seen by sys.path I don't know why the
import statement keeps failing.

Thanks,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[dieter]

Rich Shepard  writes:
>> The project directory contains subdirectories, including gui/ (with the
>> tkinter views) and classes/ with the SQLAlchemy model.py.
> ...
> Second, in ~/.bash_profile I added two lines, the first is the project's
> root directory:
>
> PYTHONPATH=$HOME/development/bustrac
> export PYTHONPATH
>
> Testing this suggests that python is finding the path:
>
> $ python3
> Python 3.7.3 (default, Mar 26 2019, 06:40:28) [GCC 5.5.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
 import os
 os.environ["PYTHONPATH"]
> '/home/rshepard/development/bustrac'

Test this by looking at "sys.path" instead:

>>> import sys
>>> sys.path

It is "sys.path" which actually controls the import machinery.

> ...
> Do I need to specify each bustrac/ subdirectory in the PYTHONPATH? If not,
> what am I still missing?

This depends on how you make the import.

"sys.path" is typically a sequence of folders. Python's
import machinery will look in those folders for modules/paackages for its
(absolute) imports.
Thus, if you use "import " or "from  import ...",
then one of those folders should contain a module or package
named "".

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Rich Shepard]

On Thu, 16 May 2019, Rich Shepard wrote:


The project directory contains subdirectories, including gui/ (with the
tkinter views) and classes/ with the SQLAlchemy model.py.


Getting closer, but still missing a piece of the solution.

First, I added __init__.py to each module subdirectory to specify that the
subdirectory is a package.

Second, in ~/.bash_profile I added two lines, the first is the project's
root directory:

PYTHONPATH=$HOME/development/bustrac
export PYTHONPATH

Testing this suggests that python is finding the path:

$ python3
Python 3.7.3 (default, Mar 26 2019, 06:40:28) 
[GCC 5.5.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

import os
os.environ["PYTHONPATH"]

'/home/rshepard/development/bustrac'

but the import is still not working.

Do I need to specify each bustrac/ subdirectory in the PYTHONPATH? If not,
what am I still missing?

Regards,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Rich Shepard]

On Fri, 17 May 2019, Inada Naoki wrote:


This is slightly off topic (not relating to your problem), but please
don't think "Python 3 doesn't require __init__.py for packages". It is
common misunderstanding.


Inada,

Actually, your response is on topic and probably the reason I have the
import problem.


Package directory without __init__.py is "namespace package".
Namespace package is not a regular package.  See [1] for detail.


Thank you for correcting me.

Regards,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Inada Naoki]

2019年5月17日(金) 9:25 Rich Shepard :
>
> My understanding is that Python3 does not require subdirectories to have an
> __init__.py file, only Python2 does. If that's not correct I'll add the
> blank file.
>

This is slightly off topic (not relating to your problem), but please don't
think "Python 3 doesn't require __init__.py for packages".
It is common misunderstanding.

Package directory without __init__.py is "namespace package".
Namespace package is not a regular package.  See [1] for detail.

[1]: https://packaging.python.org/guides/packaging-namespace-packages/

There are some difference between regular package and namespace package.
They have different module search order.  Testing tools including unittest
doesn't search namespace package.  (If they do, they may take minutes
to search million files in `node_modules` directory in your project!)
So abusing namespace package will bite you at some point.


Regards,
-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[dieter]

Rich Shepard  writes:
> I'm developing a Python3 application using Python3-3.7.3 and
> virtualenv-16.5.0 on a Slackware-14.2 host.
>
> The project directory contains subdirectories, including gui/ (with the
> tkinter views) and classes/ with the SQLAlchemy model.py.
>
> Within the gui/ directory as the cwd testing the code for a view needs to
> import the 'model' module from the class/ subdirectory and I have not been
> able to use the correct syntax to import that module. The view module starts
> with:
>
> from classes import model as m
> import data_entry_validators
> from commonDlgs import LabelInput
>
> import tkinter as tk
> from tkinter import ttk
> from datetime import datetime
>
> When I try to display this UI view I get an error:
>
> $ python3 test_act_de.py Traceback (most recent call last):
>   File "test_act_de.py", line 1, in 
> from classes import model as m
> ModuleNotFoundError: No module named 'classes'

Recently, there has been a long discussion in this list about a similar
topic. Use your favorit search engine to search for it.

The likely cause for your problem is that viewed from the location
of "test_act_de.py", "classes" is not a subdirectory.
Either put your scripts in the top level folder of your project
(such that "classes" and friends are subdirectories of the script
containing folder) or extend "sys.path" in your scripts (before
the imports) to contain this folder.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[Rich Shepard]

On Fri, 17 May 2019, duncan smith wrote:


You could make the subdirectories Python packages. Google (or better
DuckDuckGo) is your friend.


Duncan,

My understanding is that Python3 does not require subdirectories to have an
__init__.py file, only Python2 does. If that's not correct I'll add the
blank file.

Rich
--
https://mail.python.org/mailman/listinfo/python-list

Re: Import module from a different subdirectory

[duncan smith]

On 16/05/2019 22:50, Rich Shepard wrote:
> I'm developing a Python3 application using Python3-3.7.3 and
> virtualenv-16.5.0 on a Slackware-14.2 host.
> 
> The project directory contains subdirectories, including gui/ (with the
> tkinter views) and classes/ with the SQLAlchemy model.py.
> 
> Within the gui/ directory as the cwd testing the code for a view needs to
> import the 'model' module from the class/ subdirectory and I have not been
> able to use the correct syntax to import that module. The view module
> starts
> with:
> 
> from classes import model as m
> import data_entry_validators
> from commonDlgs import LabelInput
> 
> import tkinter as tk
> from tkinter import ttk
> from datetime import datetime
> 
> When I try to display this UI view I get an error:
> 
> $ python3 test_act_de.py Traceback (most recent call last):
>   File "test_act_de.py", line 1, in 
>     from classes import model as m
> ModuleNotFoundError: No module named 'classes'
> 
> Of course, 'classes' is a subdirectory not a module. My web searches have
> not found the proper syntax to import the 'model' module from the separate
> subdirectory classes and I need to learn how to do this.
> 
> Regards,
> 
> Rich

You could make the subdirectories Python packages. Google (or better
DuckDuckGo) is your friend.

Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list