[issue37089] `import Lib.os` works on windows (but shouldn't)

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Steve Dower]

Steve Dower  added the comment:

There are certainly people relying on sys.prefix being in sys.path (for .pth 
files and embedding), but probably not so many that we couldn't remove it in a 
major release. It is a bit strange in a venv, where there's typically nothing 
in the root of the environment (when we discovered this, we decided we could 
stop copying all the modules over, as they weren't being used anyway).

I think in every supported distribution we make though, there's no need for it. 
But we'd probably have to just remove it and see what breaks - most of the 
people relying on it probably have no idea, but they're also probably bundling 
it in some way that makes it easy to detect and fix.

All that said, I'm not particularly offended by this. It's namespace packages 
working as intended, and since it seems to be a highly complex interaction of 
configuration settings to get sys.prefix on sys.path in a variety of 
circumstances (we're not relying on registry settings here any more) I'd want 
to see changes and tests before assuming we can do it without regressing real 
scenarios.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Eryk Sun]

Eryk Sun  added the comment:

sys.prefix is usually argv0_path, but in a virtual environment argv0_path is 
set from "home" in pyvenv.cfg in order to find the standard library, and, if 
the site module is imported, sys.prefix is set to the virtual environment 
directory. Since for Windows site.getsitepackages() also includes sys.prefix, 
and since argv0_path and sys.prefix are different in this case, sys.path ends 
up with both of them. AFAIK, it needs neither of them.

Adding argv0_path to sys.path is a vestige of the NT installation scheme in the 
1990s. The old scheme had a "Bin" directory that contained "python.exe" and I 
presume the Python DLL and standard-library extension modules. The old setup.py 
installer [1] had the following code to set the value of the "PythonPath" 
registry key:

path = []
pwd = nt.getcwd()
for i in ["Bin",
  "Lib",
  "Lib\\win",
  "Lib\\tkinter",
  "Lib\\test",
  "Lib\\dos_8x3"]:
i = pwd + "\\" + i
path.append(i)
sys.path[1:] = path
pathvalue = strop.join(path, ";")
#print "Setting PythonPath to", pathvalue
win32api.RegSetValue(corehandle, "PythonPath", win32con.REG_SZ, pathvalue)

Note the presence of the "Bin" directory in the list.

Early NT releases used PC/getpath_nt.c [2], which was inconsistent with 
Modules/getpath.c in several ways. PC/getpathp.c was added in 1997. Early on, 
code was added to PC/getpathp.c to include the application directory in 
sys.path [3] in order to "make it work on NT". This was before it supported the 
registry "PythonPath" value that would have included the "Bin" directory.

[1] 
https://github.com/python/cpython/blob/8f1b6519803ffa7aef45beb58bda654533cb1fa8/PC/setup_nt/setup.py
[2] 
https://github.com/python/cpython/blob/8f1b6519803ffa7aef45beb58bda654533cb1fa8/PC/getpath_nt.c
[3] 
https://github.com/python/cpython/commit/8f1b6519803ffa7aef45beb58bda654533cb1fa8

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Guido van Rossum]

Guido van Rossum  added the comment:

I doubt it's meant as a feature. I don't recall anything about it. I suspect 
it's got to do with limitations of the Windows installer (at the time). Maybe 
there are other things that are siblings of "Lib" that must be importable, e.g. 
DLLs?

I recommend taking this up with Steve Dower, who owns the Windows installers 
these days.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Terry J. Reedy]

Terry J. Reedy  added the comment:

If .../Tools and its subdirectories had __init__.py files, the presence of 
sys.prefix on sys.path, on Windows, would allow imports from those subdirs.  
Otherwise, it only provides a second way to import stdlib modules.  This seems 
like a bit of a bug.  Guido, do you remember why we have this Windows-only 
feature?

--
nosy: +gvanrossum, terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Anthony Sottile]

Anthony Sottile  added the comment:

If I'm reading this correctly this is the relevant line for python3:

https://github.com/python/cpython/blob/29cb21ddb92413931e473eb799a02e2d8cdf4a45/PC/getpathp.c#L1068

it was added in 4d0d471a8031de90a2b1ce99c4ac4780e60b3bc9

python2 seems to do the same as well and I'm having a harder time tracing that 
code -- my guess is it comes from here: 
https://github.com/python/cpython/blob/103b8d9f9179089019ddb363ec0098b63816001b/PC/getpathp.c#L550

(so in that case, only present because `python.exe` is at the root of the 
prefix) -- but it's still showing up in the `virtualenv` case due to site 
manipulation (coming from `virtualenv`'s `site.py` I believe? trying to mimic 
that behaviour)

Hmmm not much information there but I have to get back to what I was doing 
before -- I'll see if I can't find more info on this later!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Steve Dower]

Steve Dower  added the comment:

Honestly, no idea. Because it always has been :)

If you can find when it was added, we may be able to answer that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Anthony Sottile]

Anthony Sottile  added the comment:

sys.prefix isn't on sys.path on other platforms -- why is it on windows?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Steve Dower]

Steve Dower  added the comment:

I don't think we can change the assumption that sys.prefix is in sys.path at 
this point, though technically it's not required (certainly not in 3.7).

Maybe we could create a new marker file that means "never treat this directory 
as a namespace package"?

--
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37089] `import Lib.os` works on windows (but shouldn't)

[Anthony Sottile]

New submission from Anthony Sottile :

Additionally, virtualenvs have the root of their directory on `sys.path` -- 
this is unlike posix behaviour

For example:

$ python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD6
4)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import Lib.os
>>> Lib.os.__file__
'C:\\Python37\\Lib\\os.py'
>>> Lib.os.listdir('C:\\')
['$Recycle.Bin', 'BGinfo', 'Documents and Settings', 'pagefile.sys', 'PerfLogs',
 'Program Files', 'Program Files (x86)', 'ProgramData', 'Python27', 'Python37',
'Recovery', 'swapfile.sys', 'System Volume Information', 'Users', 'Windows']

--
components: Library (Lib), Windows
messages: 343923
nosy: Anthony Sottile, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: `import Lib.os` works on windows (but shouldn't)
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com