[issue26939] android: test_functools hangs on armv7

Changes by Donald Stufft : -- pull_requests: +871 ___ Python tracker ___ ___

[issue26939] android: test_functools hangs on armv7

Changes by Xavier de Gaye : -- resolution: -> fixed stage: commit review -> resolved status: open -> closed ___ Python tracker ___

[issue26939] android: test_functools hangs on armv7

Roundup Robot added the comment: New changeset fd1718badb67 by Xavier de Gaye in branch '3.6': Issue #26939: Add the support.setswitchinterval() function to fix https://hg.python.org/cpython/rev/fd1718badb67 New changeset c5d7e46926ac by Xavier de Gaye in branch 'default': Issue #26939: Merge

[issue26939] android: test_functools hangs on armv7

Changes by Xavier de Gaye : -- stage: patch review -> commit review ___ Python tracker ___

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: Patch attached. -- assignee: -> xdegaye components: +Tests -Library (Lib) dependencies: -android: add platform.android_ver() stage: -> patch review versions: +Python 3.7 Added file: http://bugs.python.org/file45249/setswitchinterval.patch

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: The problem can be fixed by changing the switch interval to a higher value when the test is run on an Android emulator. Issue 26855 can provide a way to know when this is the case through the value of the ro.kernel.qemu property. -- dependencies:

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: pycond_timedwait_2.patch prints the delta time in microseconds when deadline is in the past. Running the patch with the following command: python -m test -m test_lru_cache_threaded -v test_functools gives these results: a) natively: deadline is never in

[issue26939] android: test_functools hangs on armv7

Changes by Xavier de Gaye : Added file: http://bugs.python.org/file43192/test_lru_cache_threaded-armv7-printf.txt ___ Python tracker ___

[issue26939] android: test_functools hangs on armv7

Changes by Xavier de Gaye : Added file: http://bugs.python.org/file43191/test_lru_cache_threaded-x86-printf.txt ___ Python tracker ___

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: > I don't know if it can fix the issue, but you may see my issue #23428: "Use > the monotonic clock for thread conditions on POSIX platforms". pthread_condattr_setclock() is declared in the Android API 21 headers. When a monotonic clock is set for

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: On a heavily loaded system, there is no guarantee that the date obtained from a call to gettimeofday() in PyCOND_TIMEDWAIT(), is not in the past when pthread_cond_timedwait() is called, as there may be a context switch in between. When this happens, a Python

[issue26939] android: test_functools hangs on armv7

STINNER Victor added the comment: I don't know if it can fix the issue, but you may see my issue #23428: "Use the monotonic clock for thread conditions on POSIX platforms". -- nosy: +haypo ___ Python tracker

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: Got now a full fledged gdb. Thread #1 loops infinitely in the "while (_Py_atomic_load_relaxed(_locked))" loop in take_gil(). Thread #2 is stuck in the MUTEX_LOCK(gil_mutex) statement in drop_gil() as can be seen by setting a breakpoint at the following line

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: Forgot to say that the backtraces for test_threading and test_importlib were obtained with the patch applied while they were hanging. -- ___ Python tracker

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: > Do you know in which precise test it hangs? They are named in the first msg of each issue, the issues are: issue #26938: android: test_concurrent_futures hangs on armv7 issue #26939: android: test_functools hangs on armv7 issue #26940: android:

[issue26939] android: test_functools hangs on armv7

Antoine Pitrou added the comment: > Still hangs at test_lru_cache_threaded with this last patch :( What about the other tests (test_importlib, test_threading)? Do you know in which precise test it hangs? (use "-v" to print test names) > There are two threads, gdb does not print the backtrace

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: The strace of test_lru_cache_threaded: ... gettimeofday({1462553238, 580860}, NULL) = 0 clock_gettime(CLOCK_REALTIME, {1462553238, 581276772}) = 0 gettimeofday({1462553238, 581672}, NULL) = 0 clock_gettime(CLOCK_REALTIME, {1462553238, 582078767}) = 0

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: > Can you post the backtraces of all threads (at least the innermost frames)? There are two threads, gdb does not print the backtrace of the other thread. My current gdb setup with a connection to a gdb-server misses some stuff (access to the loader or some

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: Still hangs at test_lru_cache_threaded with this last patch :( -- ___ Python tracker ___

[issue26939] android: test_functools hangs on armv7

Antoine Pitrou added the comment: Oops, sorry, hadn't seen the patch didn't compile. Can you tried with this new patch? (atomic_explicit2.patch) -- Added file: http://bugs.python.org/file42756/atomic_explicit2.patch ___ Python tracker

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: Android has stdatomic.h and HAVE_STD_ATOMIC is defined. The build fails with the patch, see the errors in the attached file. This is the native compilation of python, the patches in my build system are also applied to the native build so that they can be

[issue26939] android: test_functools hangs on armv7

Antoine Pitrou added the comment: Xavier, as a strating point, can you try the following patch? (atomic_explicit.patch) If it solves the issue, can you also measure the performance drop compared to unpatched (on a non-debug build)? `python -m test.pystone` should be sufficient. --

[issue26939] android: test_functools hangs on armv7

Changes by Antoine Pitrou : -- nosy: +neologix ___ Python tracker ___ ___ Python-bugs-list

[issue26939] android: test_functools hangs on armv7

Antoine Pitrou added the comment: I suspect this bug is caused by memory ordering differences between x86 and ARM (see https://en.wikipedia.org/wiki/Memory_ordering#Runtime_memory_ordering ), which is why it may not reproduce easily (may depend on the particular CPU implementation and/or

[issue26939] android: test_functools hangs on armv7

Chi Hsuan Yen added the comment: FWIW, test_functools passes on ASUS Zenfone 2 Laser ZE500KL with stock ROM (rooted) and my patchset. [1] The CPU is ARM64 and the Python is built as 32-bit ARM binaries. test_threading (#26941) and test_importlib (#26940) pass, too. test_concurrent_futures

[issue26939] android: test_functools hangs on armv7

Antoine Pitrou added the comment: Can you post the backtraces of all threads (at least the innermost frames)? Also, can you reproduce on an actual armv7 machine? Just wondering if there may be something peculiar in the emulator. -- nosy: +pitrou ___

[issue26939] android: test_functools hangs on armv7

Xavier de Gaye added the comment: The attached test_output.txt file is the corresponding gdb backtrace[1]. Interestingly, all the tests (issues #26938, #26939, #26940 and #26941) that hang on arm use sys.setswitchinterval() aggressively. [1] The backtrace is interleaved with: * Python

[issue26939] android: test_functools hangs on armv7

New submission from Xavier de Gaye: test_functools hangs on an android emulator running an armv7 system image (but not on x86) at API level 21. The test suite hangs at test_lru_cache_threaded (test.test_functools.TestLRUC) ... A cause of this problem may be related to the fact that the