[issue31030] sys.executable can be not normalized

[Eryk Sun]

Eryk Sun  added the comment:

> If it was the tests, they seem to have been fixed elsewhere.

The cited tests haven't been changed to work with non-normalized 
sys.executable, sys.prefix, etc. But it's not common to run the test suite with 
a non-normalized path such as "Lib/../build/python".

> you stated above that on Windows a \\?\ path is possible but 
> will not work for startup, and that you would open an issue 

Sorry, I don't remember the details of the bug that I mentioned. But it became 
a non-issue in 3.6+, which canonicalizes the executable path. For example:

>>> exe = r'\\?\C:\Program Files\Python36\python.exe'
>>> cmd = 'python -c "import sys; print(sys.executable)"'
>>> subprocess.call(cmd, executable=exe)
C:\Program Files\Python36\python.exe
0

Compare the latter to the output of GetModuleFileNameW(NULL, ...) in this case, 
which returns a \\?\ extended (verbatim) path:

>>> exe = r'\\?\C:\Program Files\Python36\python.exe'
>>> cmd = 'python -c "import _winapi; print(_winapi.GetModuleFileName(0))"'
>>> subprocess.call(cmd, executable=exe)
\\?\C:\Program Files\Python36\python.exe
0

> Do any of the non-normalized Linux (or *nix) executable paths fail for 
> startup?

Not that I'm aware of. AFAIK, it's just about the particular tests failing in 
this case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[Terry J. Reedy]

Terry J. Reedy  added the comment:

This issue was opened with 2 related facts of 2017:
a) sys.executable is not normalized
b) as a result, two tests failed
Serhiy then said "I don't know what is wrong: the value of sys.executable or 
the test."  If it was the tests, they seem to have been fixed elsewhere.

If it is the non-normalization of sys.executable, my concern for IDLE is that 
it starts the execution subprocess with subprocess.Popen([sys.executable, 
...]).  Could this be a reason for mysterious IDLE failures?

Eryk, you stated above that on Windows a \\?\ path is possible but will not 
work for startup, and that you would open an issue for this.  Have you?

Do any of the non-normalized Linux (or *nix) executable paths fail for startup?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[Eryk Sun]

Eryk Sun  added the comment:

In Python 3.10 in POSIX, it's still the case that executable, prefix, 
exec_prefix, base_prefix, and base_exec_prefix in the sys module do not get 
normalized. For example, in Linux:

$ .local/bin/../bin/python3.10 -c "import sys; print(sys.executable)"
/home/someone/.local/bin/../bin/python3.10

In test/test_sys.py, assertEqual(os.path.abspath(sys.executable), 
sys.executable) fails. In test/test_venv.py, assertIn('home = %s' % path, data) 
and assertEqual(out.strip(), expected.encode()) both fail, respectively in 
test_defaults and test_prefixes.

In Windows, prior to Python 3.6, this can be a problem if Python is run via 
CreateProcessW using a non-normalized path in lpApplicationName (rare, but 
possible) instead of letting the system search for the executable in 
lpCommandLine. For example:

>>> cmd = 'python -c "import sys; print(sys.executable)"'
>>> exe = r'C:\Program Files\Python35\..\Python35\python.exe'
>>> subprocess.call(cmd, executable=exe)
C:\Program Files\Python35\..\Python35\python.exe
0

It's no longer an issue in Windows with Python 3.6 and above. When getting the 
program path while initializing, the GetModuleFileNameW(NULL, ..) result gets 
canonicalized (e.g. via PathCchCanonicalizeEx). For example:

>>> exe = r'C:\Program Files\Python36\..\Python36\python.exe'
>>> subprocess.call(cmd, executable=exe)
C:\Program Files\Python36\python.exe
0

--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.6, Python 
3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[Eryk Sun]

Eryk Sun added the comment:

Terry, the Windows implementation calls GetModuleFileNameW to get the 
executable's fully-qualified path from the loader. However, depending on how 
Python is started, this could be a \\?\ path, which leads to a bug in the 
startup code that determines the prefix path based on the landmark "lib\os.py". 
Probably the easiest solution would be to normalize the executable path to 
remove the \\?\ or \\.\ prefix. This prefix isn't useful here anyway since 
Windows doesn't properly support running programs from long paths, not even in 
Windows 10. I'll open a new issue for this.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[Terry J. Reedy]

Terry J. Reedy added the comment:

On Windows, sys.executable is normalized before or during startup. 
C:\Programs\Python36>Lib\..\python -m test test_sys prints
 test_executable (__main__.SysModuleTest) ... ok

C:\Programs\Python36>Lib\..\python -c "import sys; print(sys.executable)"
C:\Programs\Python36\python.exe

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[STINNER Victor]

STINNER Victor added the comment:

Technically, /home/serhiy/py/cpython/Lib/../python is absolute and is a valid 
path. I'm not sure that it's possible to normalize sys.executable early in the 
Python initialization.

If we modify the site module to normalize sys.executable, it would mean that 
sys.executable wouldn't be normalized when using the python3 -S command line 
option. I dislike having subtle differences between site and no site.

I suggest to normalize sys.executable in the unit test.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

test_venv fails for the same cause.

$ Lib/../python -m test -m test_defaults test_venv
Run tests sequentially
0:00:00 load avg: 0.31 [1/1] test_venv
test test_venv failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_venv.py", line 103, in 
test_defaults
self.assertIn('home = %s' % path, data)
AssertionError: 'home = /home/serhiy/py/cpython/Lib/..' not found in 'home = 
/home/serhiy/py/cpython\ninclude-system-site-packages = false\nversion = 
3.7.0\n'

test_venv failed

1 test failed:
test_venv

Total duration: 166 ms
Tests result: FAILURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31030] sys.executable can be not normalized

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

test_executable in Lib/test/test_sys.py tests that sys.executable is absolute. 
But this test is too strong, it fails if sys.executable is not normalized.

$ Lib/../python -m test test_sys
Run tests sequentially
0:00:00 load avg: 0.39 [1/1] test_sys
test test_sys failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_sys.py", line 649, in 
test_executable
self.assertEqual(os.path.abspath(sys.executable), sys.executable)
AssertionError: '/home/serhiy/py/cpython/python' != 
'/home/serhiy/py/cpython/Lib/../python'
- /home/serhiy/py/cpython/python
+ /home/serhiy/py/cpython/Lib/../python
? +++


test_sys failed

1 test failed:
test_sys

Total duration: 2 sec
Tests result: FAILURE

I don't know what is wrong: the value of sys.executable or the test.

--
components: Interpreter Core, Tests
messages: 299065
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: sys.executable can be not normalized
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com