[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[miss-islington]

miss-islington  added the comment:


New changeset dbb37aac1428ddcba784d9ddc77c1708c391aa80 by Miss Islington (bot) 
in branch '3.8':
bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158)
https://github.com/python/cpython/commit/dbb37aac1428ddcba784d9ddc77c1708c391aa80


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Steve Dower]

Change by Steve Dower :


--
resolution: duplicate -> fixed
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[miss-islington]

Change by miss-islington :


--
pull_requests: +17618
pull_request: https://github.com/python/cpython/pull/18238

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Steve Dower]

Steve Dower  added the comment:


New changeset 0be3246d4f9c8eddcd55491901d95b09fe163f15 by Steve Dower (Adam 
Meily) in branch 'master':
bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158)
https://github.com/python/cpython/commit/0be3246d4f9c8eddcd55491901d95b09fe163f15


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[miss-islington]

Change by miss-islington :


--
pull_requests: +17617
pull_request: https://github.com/python/cpython/pull/18237

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Adam Meily]

Change by Adam Meily :


--
pull_requests: +17546
pull_request: https://github.com/python/cpython/pull/18159

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Adam Meily]

Change by Adam Meily :


--
pull_requests: +17545
pull_request: https://github.com/python/cpython/pull/18158

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Adam Meily]

Change by Adam Meily :


--
pull_requests: +17544
pull_request: https://github.com/python/cpython/pull/18157

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Eryk Sun]

Eryk Sun  added the comment:

> can you confirm that the fix is to just change spawn.get_executable() 
> to return sys.executable, like it was prior to the PR mentioned in 
> the other ticket?

Yes, in spawn.py, change `_python_exe = sys._base_executable` to `_python_exe = 
sys.executable`. This is the value that gets returned by spawn.get_executable().

This will restore the operation of the following code in Popen.__init__ in 
popen_spawn_win32.py:

python_exe = spawn.get_executable()

# bpo-35797: When running in a venv, we bypass the redirect
# executor and launch our base Python.
if WINENV and _path_eq(python_exe, sys.executable):
python_exe = sys._base_executable
env = os.environ.copy()
env["__PYVENV_LAUNCHER__"] = sys.executable

This code switches to running the base executable in a virtual environment, 
which is required in order to support duplicating handles to the child or 
stealing handles from the parent. And the startup code in the child process is 
made aware of virtual environment via the "__PYVENV_LAUNCHER__" environment 
variable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Adam Meily]

Adam Meily  added the comment:

OK, that makes sense.

For 3.7, I can create a PR for that corrects the order of arguments passed into 
_winapi.CreateProcess

For 3.8 / master, the problem appears to be that the check in 
popen_spawn_win32.py to set the subprocess env is failing because 
sys.executable != spawn.get_executable() -- spawn.get_executable() is returning 
sys._base_executable. So, can you confirm that the fix is to just change 
spawn.get_executable() to return sys.executable, like it was prior to the PR 
mentioned in the other ticket?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Eryk Sun]

Eryk Sun  added the comment:

This is a duplicate of issue 38092. The required change to fix this is very 
simple, if you want to work on a PR.

--
nosy: +eryksun
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> environment variables not passed correctly using new virtualenv 
launching in windows and python3.7+
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect

[Adam Meily]

New submission from Adam Meily :

I upgraded from Python 3.7.1 to 3.7.6 and began noticing a behavior that was 
breaking my code. My code detects if it's running in a virtualenv. This check 
worked in 3.7.1 but is broken in 3.7.6.

>From the documentation, sys.prefix and sys.exec_prefix should point to the 
>virtualenv when one is active. However, I'm seeing that both of these 
>constants are pointing to the system installation directory and not my 
>virtualenv when I am in a multiprocessing child. Here is an example output of 
>a test application running in 3.7.6 (I've attached the test script to this 
>ticket):

Parent process
=
sys.prefix:   C:\Users\user\project\venv
sys.exec_prefix:  C:\Users\user\project\venv
sys.base_prefix:  C:\Program Files\Python37
sys.base_exec_prefix: C:\Program Files\Python37
=

Subprocess
=
sys.prefix:   C:\Program Files\Python37
sys.exec_prefix:  C:\Program Files\Python37
sys.base_prefix:  C:\Program Files\Python37
sys.base_exec_prefix: C:\Program Files\Python37
=


I would expect that sys.prefix and sys.exec_prefix to be identical in the 
parent and child process.

I verified that this behavior is present in 3.7.5, 3.7.6, and 3.8.1. I am on a 
Windows 10 x64 system.

Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit 
(AMD64)] on win32

--
components: Windows
files: multiproc_venv_prefix.py
messages: 360581
nosy: meilyadam, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Multiprocessing in Virtualenv: sys.prefix is incorrect
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48862/multiproc_venv_prefix.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com