[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[Łukasz Langa]

Change by Łukasz Langa :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[miss-islington]

Change by miss-islington :


--
pull_requests: +25981
pull_request: https://github.com/python/cpython/pull/27453

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[miss-islington]

Change by miss-islington :


--
pull_requests: +25980
pull_request: https://github.com/python/cpython/pull/27452

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[hai shi]

Change by hai shi :


--
pull_requests: +25789
pull_request: https://github.com/python/cpython/pull/27240

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[hai shi]

hai shi  added the comment:

I use this add_printf_to_get_details_from_race_condition.patch to get some 
details of the running subinterpreter.

The successful output of test_already_running:
test_already_running (test.test__xxsubinterpreters.RunStringTests) ... In 
interp_create, the current state is: 0x23ee8f0
In interp_list_all, the current state is: 0x23ee8f0 # Entering _running().
Before interp runs, the current state is: 0x2489590 # Running the interp  in 
thread.
In interp_list_all, the current state is: 0x23ee8f0 # Running 
interpreters.list() in _running().
Before interp runs, the current state is: 0x23ee8f0 # Running the interp in 
test_already_running().
After interp runs, the current state is: 0x25cf0e0 # Running the interp in 
thread.
In interp_list_all, the current state is: 0x23ee8f0 # Clearing the created 
interps.

The failed output of test_already_running:
test_already_running (test.test__xxsubinterpreters.RunStringTests) ... In 
interp_create, the current state is: 0x23e58f0
In interp_list_all, the current state is: 0x23e58f0
In interp_list_all, the current state is: 0x23e58f0 # Compared to the 
successful output of test_already_running, the interp doesn't run in time. 
Before interp runs, the current state is: 0x23e58f0
spam
After interp runs, the current state is: 0x24fdfc0
In interp_list_all, the current state is: 0x23e58f0
FAIL

==
FAIL: test_already_running (test.test__xxsubinterpreters.RunStringTests)
--
Traceback (most recent call last):
  File "/home/shihai/cpython/Lib/test/test__xxsubinterpreters.py", line 835, in 
test_already_running
with self.assertRaises(RuntimeError):
AssertionError: RuntimeError not raised

--

Ran 123 tests in 71.968s

FAILED (failures=1, skipped=6)
Warning -- Uncaught thread exception: RuntimeError
Exception in thread Thread-8 (run):
Traceback (most recent call last):
  File "/home/shihai/cpython/Lib/threading.py", line 1009, in _bootstrap_inner
self.run()
^^
  File "/home/shihai/cpython/Lib/threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
^
  File "/home/shihai/cpython/Lib/test/test__xxsubinterpreters.py", line 49, in 
run
interpreters.run_string(interp, dedent(f"""
RuntimeError: unrecognized interpreter ID 60
test test__xxsubinterpreters failed.


According to above details, there are several possible explanations:
1.Some race condition break the opcode execution order.
  I checked the eval function. I didn't find that any condition will disrupt 
the execution order.
2.The running thread hasn't get the GIL in time.
  "RuntimeError: unrecognized interpreter ID 60" proves that the interp has 
been executed in _running(), but later than the test_already_running. So I 
creates PR 26598. But I catch the error again with PR 26598.
3.the running thread can't get the time slice of CPU.
  By running python with add_printf_to_get_details_from_race_condition.patch, I 
find that the thread created in _running() has be executed, but the interp 
hasn't been executed immediately in run(). At the same time, the done.set() has 
been executed in run(), so I guess the real reason is the thread created in 
_running() hasn't get the time slice of CPU to run interp in run() in 
multiprocess.

Pls correct me if I am wrong.

--
Added file: 
https://bugs.python.org/file50160/add_printf_to_get_details_from_race_condition.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[hai shi]

Change by hai shi :


--
pull_requests: +25320
pull_request: https://github.com/python/cpython/pull/26733

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[Roundup Robot]

Change by Roundup Robot :


--
nosy: +python-dev
nosy_count: 6.0 -> 7.0
pull_requests: +25305
pull_request: https://github.com/python/cpython/pull/26717

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[hai shi]

Change by hai shi :


--
pull_requests: +25182
pull_request: https://github.com/python/cpython/pull/26598

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[hai shi]

hai shi  added the comment:

OK, I try to take a look after Kyle leaves temporarily. But I haven't replicate 
this issue in my vm in recent days :(

--
nosy: +shihai1991
versions: +Python 3.11 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

Recent test_still_running() failure on AMD64 Windows8.1 Refleaks 3.10:
https://buildbot.python.org/all/#/builders/638/builds/19

1:22:17 load avg: 3.00 [393/427/2] test__xxsubinterpreters failed -- running: 
test_pydoc (9 min 7 sec), test_venv (2 min 54 sec), test_dbm_dumb (1 min 12 sec)
beginning 6 repetitions
123456
.Warning -- Uncaught thread exception: RuntimeError
Exception in thread Thread-16 (run):
Traceback (most recent call last):
  File "D:\buildarea\3.10.ware-win81-release.refleak\build\lib\threading.py", 
line 1006, in _bootstrap_inner
self.run()
  File "D:\buildarea\3.10.ware-win81-release.refleak\build\lib\threading.py", 
line 943, in run
self._target(*self._args, **self._kwargs)
  File 
"D:\buildarea\3.10.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 46, in run
interpreters.run_string(interp, dedent(f"""
RuntimeError: unrecognized interpreter ID 125
test test__xxsubinterpreters failed -- Traceback (most recent call last):
  File 
"D:\buildarea\3.10.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 753, in test_still_running
self.assertTrue(interpreters.is_running(interp),
AssertionError: False is not true : Interp 125 should be running before 
destruction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

The test still fails randomly: AMD64 Windows8.1 Refleaks 3.8
https://buildbot.python.org/all/#/builders/200/builds/48

test_subinterpreter failed twice on Refleaks which runs tests 6 times in a row.

First run:

0:33:58 load avg: 4.78 [204/423/1] test__xxsubinterpreters failed -- running: 
test_multiprocessing_spawn (12 min 42 sec), test_io (7 min 28 sec), 
test_tarfile (39.2 sec)
beginning 6 repetitions
123456
...Exception in thread Thread-38:
Traceback (most recent call last):
  File "D:\buildarea\3.8.ware-win81-release.refleak\build\lib\threading.py", 
line 932, in _bootstrap_inner
self.run()
  File "D:\buildarea\3.8.ware-win81-release.refleak\build\lib\threading.py", 
line 870, in run
self._target(*self._args, **self._kwargs)
  File 
"D:\buildarea\3.8.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 51, in run
interpreters.run_string(interp, dedent(f"""
RuntimeError: unrecognized interpreter ID 250
test test__xxsubinterpreters failed -- Traceback (most recent call last):
  File 
"D:\buildarea\3.8.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 495, in test_subinterpreter
self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

Second run:

FAIL: test_subinterpreter (test.test__xxsubinterpreters.IsRunningTests)
--
Traceback (most recent call last):
  File 
"D:\buildarea\3.8.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 495, in test_subinterpreter
self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

AMD64 Windows8.1 Refleaks 3.x:
https://buildbot.python.org/all/#/builders/157/builds/218

2:51:27 load avg: 3.54 [275/423/2] test__xxsubinterpreters crashed (Exit code 
3221225477) -- running: test_regrtest (2 hour 49 min), test_distutils (1 min 56 
sec), test_pydoc (1 min 36 sec)
beginning 6 repetitions
123456
...Windows fatal exception: access violation

Thread 0x04dc (most recent call first):
  File 
"D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\test__xxsubinterpreters.py",
 line 766 in setUp
  File 
"D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\case.py", line 
547 in _callSetUp
  File 
"D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\case.py", line 
589 in run
  File 
"D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\case.py", line 
653 in __call__
  (...)

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[Kubilay Kocak]

Kubilay Kocak  added the comment:

Failure on AMD64 FreeBSD Shared 3.x (full log attached):

==
FAIL: test_subinterpreter (test.test__xxsubinterpreters.IsRunningTests)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test__xxsubinterpreters.py",
 line 478, in test_subinterpreter
self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

...

2 tests failed:
test__xxsubinterpreters test_interpreters

...

test_already_running (test.test_interpreters.TestInterpreterRun) ... spam
Stderr:
Exception in thread Thread-19:
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/threading.py", line 
950, in _bootstrap_inner
self.run()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/threading.py", line 
888, in run
self._target(*self._args, **self._kwargs)
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_interpreters.py",
 line 45, in run
interp.run(dedent(f"""
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/support/interpreters.py",
 line 94, in run
_interpreters.run_string(self._id, src_str)
RuntimeError: unrecognized interpreter ID 111
FAIL

...

test test_interpreters failed
test_send_recv_same_interpreter (test.test_interpreters.TestSendRecv) ... ok
==
FAIL: test_still_running (test.test_interpreters.TestInterpreterDestroy)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_interpreters.py",
 line 309, in test_still_running
interp.close()
AssertionError: RuntimeError not raised
==
FAIL: test_subinterpreter (test.test_interpreters.TestInterpreterIsRunning)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_interpreters.py",
 line 207, in test_subinterpreter
self.assertTrue(interp.is_running())
AssertionError: False is not true
==
FAIL: test_already_running (test.test_interpreters.TestInterpreterRun)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_interpreters.py",
 line 383, in test_already_running
self.interp.run('print("spam")')
AssertionError: RuntimeError not raised
Stderr:
Exception in thread Thread-19:
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/threading.py", line 
950, in _bootstrap_inner
self.run()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/threading.py", line 
888, in run
self._target(*self._args, **self._kwargs)
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_interpreters.py",
 line 45, in run
interp.run(dedent(f"""
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/support/interpreters.py",
 line 94, in run
_interpreters.run_string(self._id, src_str)
RuntimeError: unrecognized interpreter ID 111
--
Ran 40 tests in 20.251s
FAILED (failures=3)
1 test failed again:
test_interpreters
== Tests result: FAILURE then FAILURE ==

...

1 test failed:
test_interpreters

...

2 re-run tests:
test__xxsubinterpreters test_interpreters

--
nosy: +koobs
Added file: 
https://bugs.python.org/file49224/3.x.koobs-freebsd-564d-build-973-test-stdio.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] [subinterpreters] test__xxsubinterpreters fails randomly

[STINNER Victor]

Change by STINNER Victor :


--
components: +Subinterpreters
title: test__xxsubinterpreters fails randomly -> [subinterpreters] 
test__xxsubinterpreters fails randomly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com