[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
Change by Anselm Kruis : -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
Anselm Kruis added the comment: The root cause for the reference leak is the global set threading._shutdown_locks. It contains Thread._tstate_lock locks of non-daemon threads. If a non-daemon thread terminates and no other thread joins the terminated thread, the _tstate_lock remains in threading._shutdown_locks forever. I could imagine that a long running server could accumulate many locks in threading._shutdown_locks over time. Therefore the leak should be fixed. There are probably several ways to deal with this issue. A straight forward approach is to discard the lock from within `tstate->on_delete` hook, that is function "void release_sentinel(void *)" in _threadmodule.c. Pull request (GH-15175) implements this idea. Eventually I should add another C-Python specific test-case to the PR. -- ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
Change by Anselm Kruis : -- pull_requests: +14906 stage: -> patch review pull_request: https://github.com/python/cpython/pull/15175 ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
New submission from Anselm Kruis : Starting with commit 468e5fec (bpo-36402: Fix threading._shutdown() race condition (GH-13948)) the following trivial test case leaks one reference and one memory block. class MiscTestCase(unittest.TestCase): def test_without_join(self): # Test that a thread without join does not leak references. # Use a debug build and run "python -m test -R: test_threading" threading.Thread().start() Attached is a patch, that adds this test case to Lib/test/test_threading.py. After you apply this patch "python -m test -R: test_threading" leaks one (additional) reference. This leak is also present in Python 3.7.4 and 3.8. I'm not sure, if it correct not to join a thread, but it did work flawlessly and didn't leak in previous releases. I didn't analyse the root cause yet. -- components: Library (Lib) files: threading-leak-test-case.diff keywords: patch messages: 349173 nosy: anselm.kruis priority: normal severity: normal status: open title: fix for bpo-36402 (threading._shutdown() race condition) causes reference leak type: resource usage versions: Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file48534/threading-leak-test-case.diff ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash
Anselm Kruis added the comment: Closing the issue is perfectly OK for me. I just want to document the problem, because there is no bpo ticket yet and Tools/msi/README.txt does not mention any problems with PGO either. 3.7 and master might be affected too, but I didn't observe this issue with 3.7.2, when I did the QA testing for Stackless 3.7.2 a few days ago. Otherwise a PGO optimized build seems to work quite well. -- ___ Python tracker <https://bugs.python.org/issue35804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash
Change by Anselm Kruis : -- nosy: +steve.dower ___ Python tracker <https://bugs.python.org/issue35804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash
Change by Anselm Kruis : Added file: https://bugs.python.org/file48072/test_ctypes.log ___ Python tracker <https://bugs.python.org/issue35804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash
New submission from Anselm Kruis :
During the QA for Stackless 3.6.8 I observed a crash in _ctypes compiled for
win32 with PGO, that also exists with plain C-Python v3.6.8. I didn't check
other versions yet.
OS: Win7 (64bit)
Compiler: Visual Studio 2017 professional 15.9.5
How to reproduce
1. Checkout v3.6.8
I'm using the git-bash
$ git status
HEAD detached at v3.6.8
nothing to commit, working tree clean
2. Clean the sandbox and compile. It is sufficient to use a short pgo-job, but
the default pgo-job works a well (but takes much more time).
$ cd PCbuild/
$ rm -rf obj win32 amd64
$ PYTHON=/e/Pythons/3.6.4-64/python.exe cmd //c build.bat --pgo-job "-m test
--pgo test_ctypes" 2>&1 | tee build.log
The file "build.log" is attached. Nothing conspicuous in it.
3. Run the test case ctypes.test.test_win32.WindowsTestCase.test_callconv_1
Sometimes the test passes, sometimes it fails with a Segmentation Fault.
$ win32/python.exe -X faulthandler -m ctypes.test.test_win32
WindowsTestCase.test_callconv_1
.
--
Ran 1 test in 0.000s
OK
$ win32/python.exe -X faulthandler -m ctypes.test.test_win32
WindowsTestCase.test_callconv_1
Windows fatal exception: access violation
Current thread 0x1574 (most recent call first):
File "C:\build\python36\lib\unittest\case.py", line 178 in handle
File "C:\build\python36\lib\unittest\case.py", line 733 in assertRaises
File "C:\build\python36\lib\ctypes\test\test_win32.py", line 20 in
test_callconv_1
File "C:\build\python36\lib\unittest\case.py", line 605 in run
File "C:\build\python36\lib\unittest\case.py", line 653 in __call__
File "C:\build\python36\lib\unittest\suite.py", line 122 in run
File "C:\build\python36\lib\unittest\suite.py", line 84 in __call__
File "C:\build\python36\lib\unittest\suite.py", line 122 in run
File "C:\build\python36\lib\unittest\suite.py", line 84 in __call__
File "C:\build\python36\lib\unittest\runner.py", line 176 in run
File "C:\build\python36\lib\unittest\main.py", line 256 in runTests
File "C:\build\python36\lib\unittest\main.py", line 95 in __init__
File "C:\build\python36\lib\ctypes\test\test_win32.py", line 165 in
File "C:\build\python36\lib\runpy.py", line 85 in _run_code
File "C:\build\python36\lib\runpy.py", line 193 in _run_module_as_main
Segmentation fault
4. I observed another variant of the crash, if I run all tests in test_ctypes
$ cmd //c rt.bat -q -v test_ctypes 2>&1 | tee test_ctypes.log
The file "test_ctypes.log" is attached. Relevant content:
test_callconv_1 (ctypes.test.test_win32.WindowsTestCase) ... XXX lineno: 124,
opcode: 0
ERROR
test_callconv_2 (ctypes.test.test_win32.WindowsTestCase) ... XXX lineno: 124,
opcode: 0
ERROR
test_variant_bool (ctypes.test.test_wintypes.WinTypesTest) ... test test_ctypes
failed
ok
==
ERROR: test_callconv_1 (ctypes.test.test_win32.WindowsTestCase)
--
Traceback (most recent call last):
File "C:\build\python36\lib\ctypes\test\test_win32.py", line 27, in
test_callconv_1
self.assertRaises(ValueError, IsWindow, 0, 0, 0)
File "C:\build\python36\lib\unittest\case.py", line 733, in assertRaises
return context.handle('assertRaises', args, kwargs)
File "C:\build\python36\lib\unittest\case.py", line 157, in handle
if not _is_subtype(self.expected, self._base_type):
File "C:\build\python36\lib\unittest\case.py", line 124, in _is_subtype
if isinstance(expected, tuple):
SystemError: unknown opcode
==
ERROR: test_callconv_2 (ctypes.test.test_win32.WindowsTestCase)
--
Traceback (most recent call last):
File "C:\build\python36\lib\ctypes\test\test_win32.py", line 36, in
test_callconv_2
self.assertRaises(ValueError, IsWindow, None)
File "C:\build\python36\lib\unittest\case.py", line 733, in assertRaises
return context.handle('assertRaises', args, kwargs)
File "C:\build\python36\lib\unittest\case.py", line 157, in handle
if not _is_subtype(self.expected, self._base_type):
File "C:\build\python36\lib\unittest\case.py", line 124, in _is_subtype
if isinstance(expected, tuple):
SystemError: unknown opcode
--
I had a quick look at _call_function_pointer() in Modules/_ctypes/callproc.c,
but I didn't see anything obvious. A very speculative first guess is the
calling convention of ffi_call() or a related funct
[issue33711] Could not find externals/db-* in msi.py on license generation
Anselm Kruis added the comment: I'm build msi installers regularly and I can confirm the issue. PR 7241 is ok and should be merged. -- nosy: +anselm.kruis ___ Python tracker <https://bugs.python.org/issue33711> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33931] Building 2.7 on Windows with PC\VS9.0 is broken
Change by Anselm Kruis : -- keywords: +patch pull_requests: +7454 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue33931> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33931] Building 2.7 on Windows with PC\VS9.0 is broken
New submission from Anselm Kruis : Currently 2.7 fails to build on Win32 using PC\VS90\build.bat -e for two reasons: 1. Wrong openssl version: PC\VS9.0 is still at version 1.0.2k, whereas PCbuild/get_externals.bat downloads version 1.0.2o. 2. Building tcl fails with well known "nmakehlp" not found problem. It is caused by the following line in PC\VS90\build.bat: "nmake -f makefile.vc MACHINE=%machine% DEBUG=%debug_flag% INSTALLDIR="%tcltkdir%" clean all". Separate invocations of nmake for "clean" and "all" fix the problem. I'll provide a pull request for both issues. I hope it is OK to make just a single PR, because the changes are trivial. Relevant output from PC\VS90\build.bat: Setting environment for using Microsoft Visual Studio 2008 x86 tools. Microsoft (R) Program Maintenance Utility Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. === *** Compiler has 'Optimizations' *** Compiler does not have 'Pentium 0x0f fix' *** Linker has 'Win98 alignment problem' *** Intermediate directory will be '.\Release_VC9\tcl_Dynamic' *** Output directory will be '.\Release_VC9' *** Suffix for binaries will be '' *** Optional defines are '-DTCL_CFGVAL_ENCODING=\"cp1252\" -DSTDC_HEADERS -DNDEBUG -DTCL_CFG_OPTIMIZED' *** Compiler version 9. Target machine is IX86 *** Host architecture is AMD64 *** Compiler options '-W3 -Ot -Oi -fp:strict -Gs -GS -GL -RTC1 -W3' *** Link options '-ltcg' *** Dependency rules are not being used. Cleaning .\Release_VC9\tcl_Dynamic\* ... Cleaning ..\win\nmakehlp.obj ... Cleaning ..\win\nmakehlp.exe ... Cleaning ..\win\_junk.pch ... Cleaning ..\win\vercl.x ... Cleaning ..\win\vercl.i ... Cleaning ..\win\versions.vc ... cl -nologo -c -W3 -W3 -D _CRT_SECURE_NO_DEPRECATE -D _CRT_NONSTDC_NO_DEPRECATE -Fp.\Release_VC9\tcl_Dynamic\ -O2 -Ot -Oi -fp:strict -Gs -GS -GL -MD -I" ..\win" -I"..\generic" -I"..\libtommath" -DTCL_PIPE_DLL=\"tclpip85.dll\" -DTCL_TOMMATH -DMP_PREC=4 -Dinline=__inline -DTCL_CFGVAL_ENCODING=\"cp1252\" -DSTDC_HEA DERS -DNDEBUG -DTCL_CFG_OPTIMIZED -DTCL_USE_STATIC_PACKAGES=0 -Fo.\Release_VC9\tcl_Dynamic\tclAppInit.obj ..\win\tclAppInit.c tclAppInit.c Der Befehl "nmakehlp" ist entweder falsch geschrieben oder konnte nicht gefunden werden. NMAKE : fatal error U1077: 'nmakehlp' : return code '0x1' Stop. Microsoft (R) Program Maintenance Utility Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. === *** Compiler has 'Optimizations' *** Compiler does not have 'Pentium 0x0f fix' *** Linker has 'Win98 alignment problem' *** Intermediate directory will be '.\Release_VC9\tcl_Dynamic' *** Output directory will be '.\Release_VC9' *** Suffix for binaries will be '' *** Optional defines are '-DTCL_CFGVAL_ENCODING=\"cp1252\" -DSTDC_HEADERS -DNDEBUG -DTCL_CFG_OPTIMIZED' *** Compiler version 9. Target machine is IX86 *** Host architecture is AMD64 *** Compiler options '-W3 -Ot -Oi -fp:strict -Gs -GS -GL -RTC1 -W3' *** Link options '-ltcg' *** Dependency rules are not being used. Installing to 'D:\kruis_F\fg2\stackless\python\PC\VS9.0\..\..\externals\tcltk' Installing tcl85.dll Datei tcl85.dll nicht gefunden NMAKE : fatal error U1077: 'xcopy' : return code '0x4' Stop. -- assignee: christian.heimes components: SSL, Tkinter messages: 320172 nosy: anselm.kruis, christian.heimes, steve.dower, zach.ware priority: normal severity: normal status: open title: Building 2.7 on Windows with PC\VS9.0 is broken type: compile error versions: Python 2.7 ___ Python tracker <https://bugs.python.org/issue33931> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30028] make test.support.temp_cwd() fork-safe
Change by Anselm Kruis : -- pull_requests: +5604 ___ Python tracker <https://bugs.python.org/issue30028> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30028] make test.support.temp_cwd() fork-safe
Change by Anselm Kruis : -- pull_requests: +5603 ___ Python tracker <https://bugs.python.org/issue30028> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31890] Please define the flag METH_STACKLESS for Stackless Python
Anselm Kruis added the comment: Hi Victor, The project is still somewhat alive, because the software, that is used to control a large percentage of the CAE simulation jobs of large German automakers, requires Stackless Python, especially the option to serialize tasklets and move them around between different computers. The Stackless project recently released Stackless Python 3.5.4 and 2.7.14 and C-Python is merged up to 3.6.1. See https://github.com/stackless-dev/stackless/tree/3.6-slp and https://bitbucket.org/stackless-dev/stackless/wiki/Download. Under the hood we fixed the reference counting (2016) and migrated from bitbucket/hg to github/git. Regards Anselm -- ___ Python tracker <https://bugs.python.org/issue31890> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31890] Please define the flag METH_STACKLESS for Stackless Python
Change by Anselm Kruis : -- keywords: +patch pull_requests: +4128 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31890> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31890] Please define the flag METH_STACKLESS for Stackless Python
New submission from Anselm Kruis : The header Include/methodobject.h defines ml_flags METH_xxx. Stackless Python adds the flag METH_STACKLESS. Traditionally Stackless used bit 0x0080 for METH_STACKLESS, but starting with C-Python 3.6 bit 0x0080 is used for METH_FASTCALL. In order to prevent future conflicts, I propose to add METH_STACKLESS to methodobject.h. #ifdef STACKLESS #define METH_STACKLESS 0x0100 #else #define METH_STACKLESS 0x #endif Include/object.h already contains a similar definition. -- components: Interpreter Core messages: 305164 nosy: anselm.kruis priority: normal severity: normal status: open title: Please define the flag METH_STACKLESS for Stackless Python type: enhancement versions: Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue31890> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31835] _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used
New submission from Anselm Kruis : Just a minor performance issue. The C functions _PyFunction_FastCallDict() and _PyFunction_FastCallKeywords() (branch 'master', Objects/call.c) and their predecessors fast_function() and _PyFunction_FastCallDict() in Python/ceval.c all contain the following sub-expression in the "if"-statement for the fast-path. For instance Objects/call.c:318 co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE) Now, if co_flags has any of the CO_FUTURE_... bits set, the expression is always False and the fast path is not used. Currently this affects only Python 3.6 and Python 2.7, because other Python versions do not use the __future__ mechanism. The fix is simple. Replace the faulty sub-expression by (co->co_flags & (~PyCF_MASK)) == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) I discovered this issue while debugging reference leaks in Stackless Python a few month ago. It is hard to write a test case, but one can compare C call stacks using a debugger. $ ulimit -c unlimited # enable core dumps $ python3.6 -c 'from __future__ import generator_stop; import os; (lambda: os.abort())()' $ gdb -batch -ex bt python3.6 core > trace_with_future $ python3.6 -c 'import os; (lambda: os.abort())()' $ gdb -batch -ex bt python3.6 core > trace_without_future If you compare the traces, the difference is in stack frame #9. Same for python2.7. -- components: Interpreter Core messages: 304702 nosy: anselm.kruis priority: normal severity: normal status: open title: _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used type: performance versions: Python 2.7, Python 3.6 ___ Python tracker <https://bugs.python.org/issue31835> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31662] trivial typos in Tools/msi/uploadrelease.bat
Change by Anselm Kruis : -- keywords: +patch pull_requests: +3830 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31662> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31662] trivial typos in Tools/msi/uploadrelease.bat
New submission from Anselm Kruis : There are 3 trivial typos in Tools/msi/uploadrelease.bat: "godo" instead of "goto" in lines 25 to 28. -- components: Build messages: 303513 nosy: anselm.kruis, steve.dower priority: normal severity: normal status: open title: trivial typos in Tools/msi/uploadrelease.bat type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue31662> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30028] make test.support.temp_cwd() fork-safe
Anselm Kruis added the comment: I had the same concerns about os.getpid(), but test.support uses it unconditionally in various places. See https://github.com/python/cpython/blob/master/Lib/test/support/__init__.py#L803 for an example. -- ___ Python tracker <http://bugs.python.org/issue30028> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30028] make test.support.temp_cwd() fork-safe
Changes by Anselm Kruis : -- pull_requests: +1212 ___ Python tracker <http://bugs.python.org/issue30028> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30028] make test.support.temp_cwd() fork-safe
New submission from Anselm Kruis: The context manager test.support.temp_cwd() creates a temporary directory and removes it on exit. The test runner test.regrtest uses this context manager. I observed an annoying behaviour of test.support.temp_cwd() on Linux/UNIX: if the code, that runs in the temp_cwd() context forks and if the forked child terminates (without calling exec), then the temporary directory will be removed twice: by the child and by the parent. This can cause errors in the parent, if the parent tries to access the no longer existing directory. I discovered this problem, when a test in test_multiprocessing_fork failed and the test directory for the complete test.regrtest-run got removed. Of course all other tests failed too. I propose to modify test.support.temp_cwd() to remove the created directory only, if the process id (os.getpid()) is unchanged. I'll create a pull request. -- components: Tests messages: 291396 nosy: anselm.kruis priority: normal severity: normal status: open title: make test.support.temp_cwd() fork-safe type: behavior versions: Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <http://bugs.python.org/issue30028> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28568] Build files in PC/VS9.0 contain an outdated sqlite version number
New submission from Anselm Kruis: Python 2.7 only. Tested with 2.7.12. Commit fa68df1d5e65 for #19450 changes the sqlite version for Python 2.7 on Windows from 3.6.21 to 3.8.11.0, but only for the build files in PCbuild. The documentation states, that the build files under PC\VS9.0 are also fully supported, but they still refer to sqlite version 3.6.21. This causes the command "PC\VS9.0\build.bat -r -e" to fail, because it first fetches sqlite 3.9.11.0 from svn.python.org and then tries to build sqlite 3.6.21, which is of course not available. The attached patch fixes the problem. -- components: Build, Windows files: fix-sqlite-version.patch keywords: patch messages: 279802 nosy: anselm.kruis, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Build files in PC/VS9.0 contain an outdated sqlite version number type: compile error versions: Python 2.7 Added file: http://bugs.python.org/file45295/fix-sqlite-version.patch ___ Python tracker <http://bugs.python.org/issue28568> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28551] sysconfig.py wrong _PROJECT_BASE for Py2.7 Windows 64bit PC/VS9.0
New submission from Anselm Kruis: Affected versions: 2.7.11, 2.7.12 Windows amd64 build with Visual Studio 2008 using the files in PC/VS9.0 If you run test_sysconfig using PC/VS9.0/rt.bat the test case test_get_config_h_filename fails. --8<---8<---8<---8<---8<---8<- D:\kruis_F\fg2\stackless64\stackless>"D:\kruis_F\fg2\stackless64\stackless\PC\VS9.0\amd64\python" -Wd -3 -E -tt "D:\kruis_F\fg2\stackless64\stackless\PC\VS9.0\ \..\..\Lib\test\regrtest.py" -v test_sysconfig == CPython 2.7.11 Stackless 3.1b3 060516 (default, Oct 28 2016, 22:32:20) [MSC v.1500 64 bit (AMD64)] == Windows-7-6.1.7601-SP1 little-endian == c:\users\kruis\appdata\local\temp\test_python_12436 Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_ site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0) [1/1] test_sysconfig test_get_config_h_filename (test.test_sysconfig.TestSysConfig) ... FAIL test_get_config_vars (test.test_sysconfig.TestSysConfig) ... ok test_get_makefile_filename (test.test_sysconfig.TestSysConfig) ... skipped 'Test is not Windows compatible' test_get_path (test.test_sysconfig.TestSysConfig) ... ok test_get_path_names (test.test_sysconfig.TestSysConfig) ... ok test_get_paths (test.test_sysconfig.TestSysConfig) ... ok test_get_platform (test.test_sysconfig.TestSysConfig) ... ok test_get_scheme_names (test.test_sysconfig.TestSysConfig) ... ok test_platform_in_subprocess (test.test_sysconfig.TestSysConfig) ... skipped 'test only relevant on MacOSX' test_symlink (test.test_sysconfig.TestSysConfig) ... skipped 'module os has no attribute symlink' test_user_similar (test.test_sysconfig.TestSysConfig) ... ok == FAIL: test_get_config_h_filename (test.test_sysconfig.TestSysConfig) -- Traceback (most recent call last): File "D:\kruis_F\fg2\stackless64\stackless\lib\test\test_sysconfig.py", line 239, in test_get_config_h_filename self.assertTrue(os.path.isfile(config_h), config_h) AssertionError: D:\kruis_F\fg2\stackless64\stackless\Include\pyconfig.h --8<---8<---8<---8<---8<---8<- The failure is caused by the migration of the build files from PCbuild to PC\VS9.0, because Lib/sysconfig.py does not know this build location. The attached patch fixes the problem. -- components: Library (Lib), Tests, Windows files: fix_sysconfig_py.patch keywords: patch messages: 279631 nosy: anselm.kruis, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: sysconfig.py wrong _PROJECT_BASE for Py2.7 Windows 64bit PC/VS9.0 type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file45256/fix_sysconfig_py.patch ___ Python tracker <http://bugs.python.org/issue28551> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23963] Windows build error using original openssl source
New submission from Anselm Kruis:
I just had an issue rebuilding 2.7.9 with openssl-1.0.1m. Because this openssl
version is not yet available from svn.python.org, I used the original source
from https://www.openssl.org/source/openssl-1.0.1m.tar.gz.
Of course Perl is installed.
Problem
The build fails, because the created makefile ms/nt.mak contains rules for
excluded ciphers ("RC5", "MDC2", "IDEA"). The documentation in
PCbuild/readme.txt anticipates a similar compile error: "You may have to
manually remove $(OBJ_D)\i_*.obj from ms\nt.mak if the build process complains
about missing files or forbidden IDEA."
Obviously manual patching of ms\nt.mak is inconvenient. A clean solution:
invoke the openssl Configure script with appropriate arguments:
perl Configure no-rc5 no-mdc2 no-idea ...
The attached patch changes PCbuild/build_ssl.py to add the no-xxx args to the
Configure command line.
Python 3.x already contains a similar fix (changeset bcf93e3766e8).
--
components: Build
files: 2.7-build_ssl.patch
keywords: patch
messages: 241096
nosy: anselm.kruis, zach.ware
priority: normal
severity: normal
status: open
title: Windows build error using original openssl source
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file39034/2.7-build_ssl.patch
___
Python tracker
<http://bugs.python.org/issue23963>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23160] Respect the environment variable SVNROOT in external-common.bat
Anselm Kruis added the comment: Your guess is correct, it will be a null merge into default. -- ___ Python tracker <http://bugs.python.org/issue23160> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23160] Respect the environment variable SVNROOT in external-common.bat
Anselm Kruis added the comment: Patch for 2.7 -- nosy: +zach.ware Added file: http://bugs.python.org/file37594/issue23160-2.7.diff ___ Python tracker <http://bugs.python.org/issue23160> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23160] Respect the environment variable SVNROOT in external-common.bat
Anselm Kruis added the comment: Patch for 3.4 -- keywords: +patch Added file: http://bugs.python.org/file37593/issue23160-3.4.diff ___ Python tracker <http://bugs.python.org/issue23160> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23160] Respect the environment variable SVNROOT in external-common.bat
New submission from Anselm Kruis: Issue #21907 introduced the environment variable SVNROOT in PCbuild/get_externals.bat. I propose to use the same variable in Tools/buildbot/external-common.bat too. This batch contains many verbatim copies of the SVN-URL. With the provided patch, it would become much simpler to use a (local) mirror of svn.python.org. -- components: Build messages: 233395 nosy: anselm.kruis priority: normal severity: normal status: open title: Respect the environment variable SVNROOT in external-common.bat type: enhancement versions: Python 2.7, Python 3.4 ___ Python tracker <http://bugs.python.org/issue23160> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17667] Windows: build with "build_pgo.bat -2" fails to optimize python.dll
Anselm Kruis added the comment: It's indeed a very low priority issue. You mention VS2008 and VS2010 PGO compiler bugs. I'm aware of the VS 2010 bugs, but I didn't observe any VS 2008 PGO bug with Python 2.7. Are you aware of any publicly available bug reports? About the black/white list. I started with a black list, but then I switched to a white list for the following reason: the goal of running the test suite is to generate a reasonable profile. It is not a goal to test anything. Therefore both white list and black listing is possible. I consider it more likely that a newly added test breaks the PGO build than that an existing test will be modified in a breaking way. Therefore I consider the while list superior for this particular application. But as you already stated, it is low prio. -- ___ Python tracker <http://bugs.python.org/issue17667> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18004] test_list.test_overflow crashes Win64
Anselm Kruis added the comment: Currently, if you run the test suite of a 64bit Windows python installation it consumes all your memory and - depending on your RAM and swap space - you system becomes unusable. That's a behaviour nobody expects. I didn't look into the implementation of the test case test_list. Therefore I can't make a proposal what to change. Obviously there are at least two options: - skip the test - replace "sys.maxint" with "sys.maxsize" in test_overflow -- ___ Python tracker <http://bugs.python.org/issue18004> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21967] Interpreter crash upon accessing frame.f_restricted of a frame from a dead thread
New submission from Anselm Kruis: If you store a reference to a frame from a dead thread and then access frame.f_restricted python eventually crashed. I didn't investigate this bug in depth. Here is my preliminary explanation. The access of frame.f_restricted causes a call of PyFrame_IsRestricted(f). PyFrame_IsRestricted is a macro and expands to ((f)->f_builtins != (f)->f_tstate->interp->builtins) Now, if the thread that created f is already dead, the f_tstate points to an invalid structure. Depending on the content of the memory the access of f_tstate->interp causes an access violation. I use a Win32 debug build to reliable reproduce the issue. -- components: Interpreter Core files: crash_on_f_restricted.py messages: 222852 nosy: anselm.kruis priority: normal severity: normal status: open title: Interpreter crash upon accessing frame.f_restricted of a frame from a dead thread type: crash versions: Python 2.7 Added file: http://bugs.python.org/file35931/crash_on_f_restricted.py ___ Python tracker <http://bugs.python.org/issue21967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21959] msi product code for 2.7.5150 not in Tools/msi/uuids.py
New submission from Anselm Kruis:
The file Tools/msi/uuids.py contains the product codes for all recently
released Python 2.x versions except 2.7.8. Without this code it is not possible
to recreate the MSI installer using Tools\msi\msi.py.
The product code of https://www.python.org/ftp/python/2.7.8/python-2.7.8.msi is
'{61121B12-88BD-4261-A6EE-AB32610A56DD}'.
By the way. We had exactly the same issue with 2.7.5: #18023
--
components: Windows
messages: 222774
nosy: anselm.kruis, loewis, zach.ware
priority: normal
severity: normal
status: open
title: msi product code for 2.7.5150 not in Tools/msi/uuids.py
type: behavior
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue21959>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21959] msi product code for 2.7.8150 not in Tools/msi/uuids.py
Changes by Anselm Kruis : -- title: msi product code for 2.7.5150 not in Tools/msi/uuids.py -> msi product code for 2.7.8150 not in Tools/msi/uuids.py ___ Python tracker <http://bugs.python.org/issue21959> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18015] python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
Anselm Kruis added the comment:
I created a small *.pth to monkey patch collections.py until 2.7.6 gets
released. Maybe this is useful for someone else. Therefore I attach it here.
The pth file runs the following code during Python startup:
import collections
def _fix_issue_18015(collections):
try:
template = collections._class_template
except AttributeError:
# prior to 2.7.4 _class_template didn't exists
return
if not isinstance(template, basestring):
return # strange
if "__dict__" in template or "__getstate__" in template:
return # already patched
lines = template.splitlines()
indent = -1
for i,l in enumerate(lines):
if indent < 0:
indent = l.find('def _asdict')
continue
if l.startswith(' '*indent + 'def '):
lines.insert(i, ' '*indent + 'def __getstate__(self): pass')
lines.insert(i, ' '*indent + '__dict__ = _property(_asdict)')
break
collections._class_template = '''\n'''.join(lines)
_fix_issue_18015(collections)
--
Added file: http://bugs.python.org/file30338/fix_python_275_issue18015.pth
___
Python tracker
<http://bugs.python.org/issue18015>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18015] python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
Anselm Kruis added the comment: Just for the records: the patch works as expected. -- ___ Python tracker <http://bugs.python.org/issue18015> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18004] test_list.test_overflow crashes Win64
Anselm Kruis added the comment: > I take it you have more than 16GB of RAM? I used a system with 16GB Ram. > What happens if you replace "sys.maxint" with "sys.maxsize" in test_overflow? The test passes. Both mul and imul raise MemoryError. -- ___ Python tracker <http://bugs.python.org/issue18004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18023] msi product code for 2.7.5150 not in Tools/msi/uuids.py
New submission from Anselm Kruis:
The file Tools/msi/uuids.py contains the product codes for all recently
released Python 2.x versions except 2.7.5. Without this code it is not possible
to recreate the MSI installer using Tools\msi\msi.py.
The product code of http://www.python.org/ftp/python/2.7.5/python-2.7.5.msi is
'{DBDD570E-0952-475F-9453-AB88F3DD5659}'.
--
components: Build, Windows
messages: 189686
nosy: anselm.kruis
priority: normal
severity: normal
status: open
title: msi product code for 2.7.5150 not in Tools/msi/uuids.py
type: compile error
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue18023>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18015] python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
New submission from Anselm Kruis:
Change 18303391b981 breaks unpickling named tuples pickled by 2.7.3 and 2.7.4.
See closed issue #15535 for the full story. Unfortunately Raymond was wrong,
when he wrote that the addition of __dict__ was a 2.7.4 change. It was added by
changeset 26d5f022eb1a in 2.7.3.
Now 2.7.5 can't unpickle any named tuples pickled by 2.7.3, which is probably
one of the most widely used python versions.
Example:
Pickle a namd tuple using 2.7.3 and unpickle it using 2.7.5.
anselm@Emmy:~$ python2.7
Python 2.7.3 (default, Sep 16 2012, 21:46:37)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
>>> import pickletools
>>> import pickle
>>> N=collections.namedtuple("N","a")
>>> n=N(1)
>>> p=pickle.dumps(n, 2)
>>> p2=pickletools.optimize(p)
>>> pickletools.dis(p2)
0: \x80 PROTO 2
2: cGLOBAL '__main__ N'
14: KBININT11
16: \x85 TUPLE1
17: \x81 NEWOBJ
18: cGLOBAL 'collections OrderedDict'
43: ]EMPTY_LIST
44: ]EMPTY_LIST
45: (MARK
46: USHORT_BINSTRING 'a'
49: KBININT11
51: eAPPENDS(MARK at 45)
52: aAPPEND
53: \x85 TUPLE1
54: RREDUCE
55: bBUILD
56: .STOP
highest protocol among opcodes = 2
>>> print repr(p2)
'\x80\x02c__main__\nN\nK\x01\x85\x81ccollections\nOrderedDict\n]](U\x01aK\x01ea\x85Rb.'
anselm@Emmy:~/sc/eclipsews/fg2py$ fg2python
Python 2.7.5 (default, May 18 2013, 17:02:17)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> import collections
>>> N=collections.namedtuple("N","a")
>>> pickle.loads('\x80\x02c__main__\nN\nK\x01\x85\x81ccollections\nOrderedDict\n]](U\x01aK\x01ea\x85Rb.')
Traceback (most recent call last):
File "", line 1, in
File
"/home/anselm/sc/eclipsews/fg2py/arch/rhel4u4-x86_64/lib/python2.7/pickle.py",
line 1419, in loads
return Unpickler(file).load()
File
"/home/anselm/sc/eclipsews/fg2py/arch/rhel4u4-x86_64/lib/python2.7/pickle.py",
line 895, in load
dispatch[key](self)
File
"/home/anselm/sc/eclipsews/fg2py/arch/rhel4u4-x86_64/lib/python2.7/pickle.py",
line 1261, in load_build
d = inst.__dict__
AttributeError: 'N' object has no attribute '__dict__'
As we can see from the trace back, the problem arises from the pickle op-code
'BUILD'. BUILD requires that the object to be build either has a method
__setstate__ or has an attribute __dict__. Therefore I propose:
- Revert change 18303391b981 and add a __getstate__ method
This is the Python 3 fix for the problem.
or
- Add a method __setstate__:
def __setstate__(self, state):
"""For compatibility with Python 2.7.3 and 2.7.4"""
pass
--
components: Library (Lib)
messages: 189614
nosy: anselm.kruis, rhettinger
priority: normal
severity: normal
status: open
title: python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
type: behavior
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue18015>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15535] Fix pickling efficiency of named tuples in 2.7.3
Anselm Kruis added the comment: >> why the different fix for 3.3 > > I reverted the 2.7.4 addition of __dict__ rather than introduce more > differences between point releases with possible unintended effects. __dict__ was a 2.7.3 addition (changeset 26d5f022eb1a). Now unpickling of named tuples created by 2.7.3 and 2.7.4 fails. -- nosy: +anselm.kruis ___ Python tracker <http://bugs.python.org/issue15535> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18004] test_list.test_overflow crashes Win64
New submission from Anselm Kruis: I installed Python 2.7.5 including tests using the MSI installer from http://www.python.org/ftp/python/2.7.5/python-2.7.5.amd64.msi Running python.exe -m test.regrtest -v test_list consumes all available memory and renders the my system completely unusable. I had to reboot. The problem is caused by the test case "test_overflow". If you run the test with an imposed memory limit, e.g. by using ulimitnt (http://code.google.com/p/ulimitnt/) , test_overflow passes. I propose to skip this test on Win64. -- components: Tests messages: 189491 nosy: anselm.kruis priority: normal severity: normal status: open title: test_list.test_overflow crashes Win64 type: resource usage versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue18004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5773] Crash on shutdown after os.fdopen(2) in debug builds
Anselm Kruis added the comment: Hi, I was faced with a very similar problem also caused by an invalid file descriptor. My solution is to set an invalid parameter handler, that does nothing. This effectively disables Dr. Watson. Perhaps this is a suitable solution for other users too. And it does not require a patch. def set_invalid_parameter_handler(flag): """ Set the MSVCRT invalid parameter handler. If flag is True, this function sets an invalid parameter handler, that does nothing. This effectively disables Dr. Watson. If flag is an integer number, it must be the address of an invalid parameter handler function. If flag is None, this function removes the invalid parameter handler. This effectively enables Dr. Watson. The return value is the address of the current handler or None, if no handler is installed. Example:: old = set_invalid_parameter_handler(True) try: do_something_nasty finally: set_invalid_parameter_handler(old) """ try: # get the msvcrt library import ctypes.util libc = ctypes.util.find_msvcrt() if not libc: # probably not windows return None libc = getattr(ctypes.cdll, libc) siph = libc._set_invalid_parameter_handler siph.restype = ctypes.c_void_p siph.argtypes = [ ctypes.c_void_p ] # now we need a suitable handler. # The handler must simply return without performing any actions. # Of course there is none. # But if we look at the calling convention (cdecl), and # at the fact, that we don't need the argument values # we find, that we can call any function, as long as the function # does not harm. A suitable function is "int abs(abs)". null_handler = libc.abs except Exception: # probably not the correct windows version return None if flag is True: flag = null_handler return siph(flag) -- nosy: +anselm.kruis ___ Python tracker <http://bugs.python.org/issue5773> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17667] Windows: build with "build_pgo.bat -2" fails to optimize python.dll
New submission from Anselm Kruis: Version: 2.7.4 (and any prior 2.7-version. not tested with 3.x) Compiler: VS 2008 Professional If I compile Python using the batch > PCbuild\build_pgo.bat -2 Visual Studio fails to correctly collect profile information for the project "python27.dll". As a result, no real optimisation takes place. The command "build_pgo.bat -2" runs the full test suite (..\lib\test\regrtest.py) to collect profile data. Experiments show, that several test cases (i.e. test_os and test_subprocess) break the collection of profile data. Probably the failure is related to the creation of sub-processes. I consider it a Visual Studio bug. To work around this issue I created a list of good test cases and patched build_pgo.bat to run only the tests from this list. -- components: Build, Windows files: build_pgo-2.patch keywords: patch messages: 186322 nosy: akruis priority: normal severity: normal status: open title: Windows: build with "build_pgo.bat -2" fails to optimize python.dll type: compile error versions: Python 2.7 Added file: http://bugs.python.org/file29739/build_pgo-2.patch ___ Python tracker <http://bugs.python.org/issue17667> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
