[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Vincent Gatine]

Vincent Gatine added the comment:

Tried your suggestion. Nothing is printed and program still crash.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[STINNER Victor]

STINNER Victor added the comment:

I'm curious. Would you mind to try to run your frozen application using 
PYTHONMALLOC=debug environment variable on a Python without the fix, to see if 
you get an error? I'm not sure that the debug hooks on memory allocators are 
set early enough to detect your specific bug.

https://docs.python.org/dev/using/cmdline.html#envvar-PYTHONMALLOC

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Benjamin Peterson]

Benjamin Peterson added the comment:


New changeset 88d0663005d258526496d1f8ee0acb7103c69e80 by Benjamin Peterson 
(Miss Islington (bot)) in branch '3.6':
[3.6] closes bpo-31532: Fix memory corruption due to allocator mix (GH-3679) 
(#3681)
https://github.com/python/cpython/commit/88d0663005d258526496d1f8ee0acb7103c69e80


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Roundup Robot]

Changes by Roundup Robot :


--
pull_requests: +3670

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Benjamin Peterson]

Benjamin Peterson added the comment:


New changeset 3d1e2ab584ed0175592b5be2a0bc98dc1723776a by Benjamin Peterson 
(nurelin) in branch 'master':
bpo-31532: Fix memory corruption due to allocator mix (#3679)
https://github.com/python/cpython/commit/3d1e2ab584ed0175592b5be2a0bc98dc1723776a


--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Vincent Gatine]

Changes by Vincent Gatine :


--
nosy: +nurelin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Vincent Gatine]

Changes by Vincent Gatine :


--
keywords: +patch
pull_requests: +3668
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31532] Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free

[Hartmut Goebel]

New submission from Hartmut Goebel:

When using

Py_GetPath();
Py_SetPath(pypath_w);

near the beginning of a program, Py_SetPath crashes with memory corruption if 
run on a systems with glibc's MALLOC_CHECK enabled.

The reason is: Py_GetPath and Py_SetPath use different memory interfaces.

* Py_GetPath() calls calculate_path(), which uses PyMem_New() to allocate the 
memory (see 
https://github.com/python/cpython/blob/v3.6.0/Modules/getpath.c#L738, assigned 
to `buf` first)

* But Py_SetPath() uses PyMem_RawFree() to free the memory (see 
https://github.com/python/cpython/blob/v3.6.0/Modules/getpath.c#L830).


This error DOES NOT occur on Windows, since for win32 there is a separate 
implementation, which uses PyMem_RawMalloc (see 
https://github.com/python/cpython/blob/v3.6.0/PC/getpathp.c#L378).

This error also DOES NOT occur in Python <= 3.5, since PYMEM_FUNCS have been 
the same as PYRAW_FUNCS (see 
https://github.com/python/cpython/blob/v3.5.0/Objects/obmalloc.c#L148). This 
was changed in v3.6.0: PYMEM_FUNCS now is PYOBJ_FUNCS, see 
https://github.com/python/cpython/blob/v3.6.0/Objects/obmalloc.c#L159.

This error only occurs when running on a system with glibc's MALLOC_CHECK 
enabled, which seems to be the default st least on CentOS, Fedora and ArchLinux.

Example code and relevant source see below.


Example stack trace


*** glibc detected *** ./dist/jcollect3: double free or corruption (out): 
0x7fde271ab030 ***
=== Backtrace: =
/lib64/libc.so.6[0x37f0675e66]
/lib64/libc.so.6[0x37f06789b3]
/tmp/_MEIhKg3kx/libpython3.6m.so.1.0(Py_SetPath+0x15)[0x7fde2d20e825]
./dist/jcollect3[0x4043e2]
./dist/jcollect3[0x402db2]
./dist/jcollect3[0x402fb0]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x37f061ed5d]
./dist/jcollect3[0x401a9e]
=== Memory map: 


How to reproduce
~

An example for this problem is the bootloader of PyInstaller (as of 2017-08-01).

* Make sure you are running on a Linux distribution having glibc's MALLOC_CHECK 
enabled.

* Set up a virtual environment (just too keep your system clean)

pip install 
https://github.com/pyinstaller/pyinstaller/archive/develop@%7B2017-08-01%7D.zip
echo 'print("Hello")' > test.py
pyinstaller test.py
dist/test/test  # run the frozen script

The relevant source of PyInstaller's bootloader is at 


--
components: Interpreter Core
messages: 302624
nosy: htgoebel
priority: normal
severity: normal
status: open
title: Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex 
with PyMem_Raw_Free
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com