[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Anselm Kruis]

Anselm Kruis  added the comment:

Closing the issue is perfectly OK for me. I just want to document the problem, 
because there is no bpo ticket yet and Tools/msi/README.txt does not mention 
any problems with PGO either.

3.7 and master might be affected too, but I didn't observe this issue with 
3.7.2, when I did the QA testing for Stackless 3.7.2 a few days ago.

Otherwise a PGO optimized build seems to work quite well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Steve Dower]

Steve Dower  added the comment:

We already compile 32-bit CPython without PGO on Windows, so I don't think 
there's anything to do here apart from record the issue.

It's very likely that 3.8 will finally get an updated libffi, which should deal 
with this completely (or force us to take another look). 3.6 is not getting 
more binary releases though, and 3.7 would need a targeted fix if it repros 
there, so all we can really do is close this issue, unless you have additional 
info on its impact?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Anselm Kruis]

Change by Anselm Kruis :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Anselm Kruis]

Change by Anselm Kruis :


Added file: https://bugs.python.org/file48072/test_ctypes.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Anselm Kruis]

New submission from Anselm Kruis :

During the QA for Stackless 3.6.8 I observed a crash in _ctypes compiled for 
win32 with PGO, that also exists with plain C-Python v3.6.8. I didn't check 
other versions yet.

OS: Win7 (64bit)
Compiler: Visual Studio 2017 professional 15.9.5

How to reproduce

1. Checkout v3.6.8

I'm using the git-bash
$ git status
HEAD detached at v3.6.8
nothing to commit, working tree clean

2. Clean the sandbox and compile. It is sufficient to use a short pgo-job, but 
the default pgo-job works a well (but takes much more time).

$ cd PCbuild/
$ rm -rf obj win32 amd64
$ PYTHON=/e/Pythons/3.6.4-64/python.exe cmd //c build.bat --pgo-job "-m test 
--pgo test_ctypes" 2>&1 | tee build.log

The file "build.log" is attached. Nothing conspicuous in it.

3. Run the test case ctypes.test.test_win32.WindowsTestCase.test_callconv_1
Sometimes the test passes, sometimes it fails with a Segmentation Fault.

$ win32/python.exe -X faulthandler -m ctypes.test.test_win32 
WindowsTestCase.test_callconv_1
.
--
Ran 1 test in 0.000s

OK

$ win32/python.exe -X faulthandler -m ctypes.test.test_win32 
WindowsTestCase.test_callconv_1
Windows fatal exception: access violation

Current thread 0x1574 (most recent call first):
  File "C:\build\python36\lib\unittest\case.py", line 178 in handle
  File "C:\build\python36\lib\unittest\case.py", line 733 in assertRaises
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 20 in 
test_callconv_1
  File "C:\build\python36\lib\unittest\case.py", line 605 in run
  File "C:\build\python36\lib\unittest\case.py", line 653 in __call__
  File "C:\build\python36\lib\unittest\suite.py", line 122 in run
  File "C:\build\python36\lib\unittest\suite.py", line 84 in __call__
  File "C:\build\python36\lib\unittest\suite.py", line 122 in run
  File "C:\build\python36\lib\unittest\suite.py", line 84 in __call__
  File "C:\build\python36\lib\unittest\runner.py", line 176 in run
  File "C:\build\python36\lib\unittest\main.py", line 256 in runTests
  File "C:\build\python36\lib\unittest\main.py", line 95 in __init__
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 165 in 
  File "C:\build\python36\lib\runpy.py", line 85 in _run_code
  File "C:\build\python36\lib\runpy.py", line 193 in _run_module_as_main
Segmentation fault


4. I observed another variant of the crash, if I run all tests in test_ctypes

$ cmd //c rt.bat -q -v test_ctypes 2>&1 | tee test_ctypes.log

The file "test_ctypes.log" is attached. Relevant content:
test_callconv_1 (ctypes.test.test_win32.WindowsTestCase) ... XXX lineno: 124, 
opcode: 0
ERROR
test_callconv_2 (ctypes.test.test_win32.WindowsTestCase) ... XXX lineno: 124, 
opcode: 0
ERROR
test_variant_bool (ctypes.test.test_wintypes.WinTypesTest) ... test test_ctypes 
failed
ok

==
ERROR: test_callconv_1 (ctypes.test.test_win32.WindowsTestCase)
--
Traceback (most recent call last):
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 27, in 
test_callconv_1
self.assertRaises(ValueError, IsWindow, 0, 0, 0)
  File "C:\build\python36\lib\unittest\case.py", line 733, in assertRaises
return context.handle('assertRaises', args, kwargs)
  File "C:\build\python36\lib\unittest\case.py", line 157, in handle
if not _is_subtype(self.expected, self._base_type):
  File "C:\build\python36\lib\unittest\case.py", line 124, in _is_subtype
if isinstance(expected, tuple):
SystemError: unknown opcode

==
ERROR: test_callconv_2 (ctypes.test.test_win32.WindowsTestCase)
--
Traceback (most recent call last):
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 36, in 
test_callconv_2
self.assertRaises(ValueError, IsWindow, None)
  File "C:\build\python36\lib\unittest\case.py", line 733, in assertRaises
return context.handle('assertRaises', args, kwargs)
  File "C:\build\python36\lib\unittest\case.py", line 157, in handle
if not _is_subtype(self.expected, self._base_type):
  File "C:\build\python36\lib\unittest\case.py", line 124, in _is_subtype
if isinstance(expected, tuple):
SystemError: unknown opcode

--


I had a quick look at  _call_function_pointer() in Modules/_ctypes/callproc.c, 
but I didn't see anything obvious. A very speculative first guess is the 
calling convention of ffi_call() or a related function written in (inline) 
assembly.

Work around: compile _ctypes for win32 without PGO.

--
components: ctypes
files: build.log
messages: 334202
nosy: anselm.kruis
priority: normal
severity: normal
status: open
title: v3.6.8 _ctypes win32 compiled with pgo crash
type: crash
versions: Python 3.6
Added