[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Ned Deily]

Ned Deily  added the comment:


New changeset 46cbf6148a46883110883488d3e9febbe46ba861 by Steve Dower in branch 
'3.6':
[3.6] bpo-29778: Ensure python3.dll is loaded from correct locations when 
Python is embedded (GH-21298) (#21354)
https://github.com/python/cpython/commit/46cbf6148a46883110883488d3e9febbe46ba861


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +20500
pull_request: https://github.com/python/cpython/pull/21354

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Ned Deily]

Ned Deily  added the comment:


New changeset 110dd153662a13b8ae1bb06348e5b1f118ab26d7 by Steve Dower in branch 
'3.7':
[3.7] bpo-29778: Ensure python3.dll is loaded from correct locations when 
Python is embedded (GH-21297) (#21298)
https://github.com/python/cpython/commit/110dd153662a13b8ae1bb06348e5b1f118ab26d7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset aa7f7756149a10c64d01f583b71e91814db886ab by Miss Islington (bot) 
in branch '3.8':
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is 
embedded (GH-21297) (GH-21352)
https://github.com/python/cpython/commit/aa7f7756149a10c64d01f583b71e91814db886ab


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[miss-islington]

miss-islington  added the comment:


New changeset 4981fe36c7477303de830e8dca929a02caaaffe4 by Miss Islington (bot) 
in branch '3.9':
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is 
embedded (GH-21297)
https://github.com/python/cpython/commit/4981fe36c7477303de830e8dca929a02caaaffe4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[miss-islington]

Change by miss-islington :


--
pull_requests: +20498
pull_request: https://github.com/python/cpython/pull/21352

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:


New changeset dcbaa1b49cd9062fb9ba2b9d49555ac6cd8c60b5 by Steve Dower in branch 
'master':
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is 
embedded (GH-21297)
https://github.com/python/cpython/commit/dcbaa1b49cd9062fb9ba2b9d49555ac6cd8c60b5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[miss-islington]

Change by miss-islington :


--
pull_requests: +20497
pull_request: https://github.com/python/cpython/pull/21351

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +20458
pull_request: https://github.com/python/cpython/pull/21306

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
pull_requests:  -20454

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +20454
pull_request: https://github.com/python/cpython/pull/21304

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:

Bumping to release blocker and adding RMs. Should definitely get this fix 
merged within the next week, and I don't want the next round of releases to go 
out without it.

--
nosy: +lukasz.langa, ned.deily
priority: normal -> release blocker
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +20447
pull_request: https://github.com/python/cpython/pull/21298

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +20446
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21297

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
assignee:  -> steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:

Sorry, I take that back. Earlier versions would indeed skip initialization in 
some cases.

I propose we deprecate the dll_path field in PathConfig and just get the path 
directly in the three places it's necessary. The path calculations have 
security exposure, so let's just avoid trying to manage additional state around 
it unnecessarily.

I'll work on a patch this week unless someone else gets to it first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:

> I understand that LoadLibraryExW() must not be attempted if _Py_dll_path is 
> empty, or if Py_GetPrefix() is empty. Am I right?

More likely those should never be empty. Perhaps sys.prefix is optional, but 
the DLL path is the current executing module, and should always be set.

I suspect you're right, that 3.7 is also affected. But earlier versions would 
only _not_ fill the DLL path for static (non-shared) builds.

It looks like Py_SetPath in 3.7 started clearing it unnecessarily, so that may 
be the cause.

--
stage: resolved -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

STINNER Victor  added the comment:

I understand that Python 3.5, 3.6 and 3.7 are also affected. It's not a 
regression.

On Python 3.5, 3.6 and 3.7, when Py_SetPath(path) is called, Py_GetPrefix() 
also returns an empty path. So at least the directory based on Py_GetPrefix() 
should also be skipped on Python 3.5-3.7 if Py_GetPrefix() is empty, right?

--
type: behavior -> security
versions: +Python 3.10, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

STINNER Victor  added the comment:

_Py_CheckPython3() tries to load "python3.dll" from two directories:

* first one based on _Py_dll_path
* second one based on Py_GetPrefix()

I understand that LoadLibraryExW() must not be attempted if _Py_dll_path is 
empty, or if Py_GetPrefix() is empty. Am I right?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

STINNER Victor  added the comment:

> Hm, I'm seeing _Py_CheckPython3() use Py_GetPrefix(), which uses 
> _Py_path_config.prefix?

Oh right, that's the initial issue:

> Later the interpreter tries to load python3.dll and uses dllpath which is 
> empty by default. This empty path gets joined with \python3.dll and 
> \DLLs\python3.dll which is used in the LoadLibraryExW resulting in loading 
> python3.dll from the root location of the windows drive the application is 
> running from.

I reopen the issue.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Anthony Wee]

Anthony Wee  added the comment:

Hm, I'm seeing _Py_CheckPython3() use Py_GetPrefix(), which uses 
_Py_path_config.prefix?

https://github.com/python/cpython/blob/c02b41b1fb115c87693530ea6a480b2e15460424/PC/getpathp.c#L1185

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

STINNER Victor  added the comment:

> It looks like Py_SetPath() sets the _Py_path_config.prefix to "", but I'm not 
> seeing anything else set it to a real value?

In the master branch, _Py_CheckPython3() doesn't use _Py_path_config.prefix.

_PyPathConfig_InitDLLPath() calls GetModuleFileNameW(PyWin_DLLhModule, 
dll_path, MAXPATHLEN) if PyWin_DLLhModule is initialized.

For example, _PyPathConfig_InitDLLPath() is called by Py_Initialize() and 
Py_SetPath().

PyWin_DLLhModule is initialized by DllMain().

The code in the 3.8 branch looks very similar (I backported my "Remove 
_PyPathConfig.dll_path" change to 3.8: commit 
9f3dcf802eefeb5ab821ce3c7204ab46557d53d7).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Anthony Wee]

Anthony Wee  added the comment:

Thank you Steve!

I'm still seeing python3.dll being loaded from \DLLs\python3.dll.

_Py_CheckPython3() uses Py_GetPrefix() as a prefix for \DLLs\python3.dll.

It looks like Py_SetPath() sets the _Py_path_config.prefix to "", but I'm not 
seeing anything else set it to a real value?

https://github.com/python/cpython/blob/7b79dc9200a19ecbac667111dffd58e314be02a8/Python/pathconfig.c#L508

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

STINNER Victor  added the comment:

Oops, I'm guilty of pushing this change! Sorry & thanks for the fix.

if (_Py_dll_path == NULL) {
/* Already set: nothing to do */
return _PyStatus_OK();
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[miss-islington]

miss-islington  added the comment:


New changeset a9a43c221bf3896ed1d1c2eee2531b7121cf78e4 by Miss Islington (bot) 
in branch '3.8':
bpo-29778: Fix incorrect NULL check in _PyPathConfig_InitDLLPath() (GH-17818)
https://github.com/python/cpython/commit/a9a43c221bf3896ed1d1c2eee2531b7121cf78e4


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[miss-islington]

Change by miss-islington :


--
pull_requests: +17287
pull_request: https://github.com/python/cpython/pull/17871

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Change by Steve Dower :


--
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:

Thanks, Anthony! And congratulations on becoming a CPython contributor!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:


New changeset 7b79dc9200a19ecbac667111dffd58e314be02a8 by Steve Dower (Anthony 
Wee) in branch 'master':
bpo-29778: Fix incorrect NULL check in _PyPathConfig_InitDLLPath() (GH-17818)
https://github.com/python/cpython/commit/7b79dc9200a19ecbac667111dffd58e314be02a8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Anthony Wee]

Change by Anthony Wee :


--
pull_requests: +17244
pull_request: https://github.com/python/cpython/pull/17818

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower  added the comment:

> It looks like there has been a regression in the fix for this issue.

You're right. Care to create a pull request to fix it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Anthony Wee]

Anthony Wee  added the comment:

It looks like there has been a regression in the fix for this issue. 

The commit below introduced a NULL check which causes a call to 
_PyPathConfig_Init() to be skipped if _Py_dll_path == NULL. It seems like the 
check should be "if (_Py_dll_path != NULL)"?

https://github.com/python/cpython/commit/c422167749f92d4170203e996a2c619c818335ea#diff-87aed37b4704d4e1513be6378c9c7fe6R169

--
nosy: +anthonywee

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed
versions: +Python 3.8 -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[STINNER Victor]

STINNER Victor  added the comment:

> When Py_SetPath is used to set up module path at initialization, the 
> Py_SetPath causes getpathp.c::calculate_path not to be called. However, 
> calculate path is the only function calling getpathp.c::get_progpath which 
> initializes the local dllpath static variable.

I fixed this issue in Python 3.8 with this commit:

commit 410759fba80aded5247b693c60745aa16906f3bb
Author: Victor Stinner 
Date:   Sat May 18 04:17:01 2019 +0200

bpo-36763: Remove _PyCoreConfig.dll_path (GH-13402)

I modified Py_SetPath() like that:

-new_config.dll_path = _PyMem_RawWcsdup(L"");
+new_config.dll_path = _Py_GetDLLPath();

Py_SetPath() no longer sets dll_path to an empty string.

Since we only got one bug report and I believe that Tibor Csonka found a way to 
workaround the issue since he reported it, I close the issue.

Please reopen/comment the issue if you would like to get this issue fixed in 
Python 3.7 as well.

--

Moreover, the PEP 587 now has a better API to configure embedded Python. I just 
implemented this PEP in bpo-36763.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Changes by Steve Dower :


--
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower added the comment:

Ah, I see. We force load it in PC/getpathp.c to ensure that it's ours and not 
another version's python3.dll.

We should probably refactor the GetModuleFileNameW call into its own function 
so we can call it from anywhere we need.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower added the comment:

I thought we'd documented that if you set the path when embedding you should 
also set the program name, but perhaps not (didn't check just now). If not, we 
should do that.

We shouldn't be loading python3.dll anywhere. Are you sure that's in CPython? 
Do you have a reference to the source file?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Tibor Csonka]

New submission from Tibor Csonka:

When Py_SetPath is used to set up module path at initialization, the Py_SetPath 
causes getpathp.c::calculate_path not to be called. However, calculate path is 
the only function calling getpathp.c::get_progpath which initializes the local 
dllpath static variable.

Later the interpreter tries to load python3.dll and uses dllpath which is empty 
by default. This empty path gets joined with \python3.dll and \DLLs\python3.dll 
which is used in the LoadLibraryExW resulting in loading python3.dll from the 
root location of the windows drive the application is running from.

The behavior was reproduced using PyInstaller but it is present in any 
embedding application which uses Py_SetPath.

--
components: Windows
messages: 289334
nosy: Tibor Csonka, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: _Py_CheckPython3 uses uninitialized dllpath when embedder sets module 
path with Py_SetPath
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com