New submission from Sam Gross <[email protected]>:
The runtest_mp.py has a race condition between checking for worker.is_alive() and processing the queue that can lead to indefinite hangs. The hang happens when the all the results from the self.output queue are processed but at least one of the workers hasn't finished exiting. https://github.com/python/cpython/blob/8d7644fa64213207b8dc6f555cb8a02bfabeced2/Lib/test/libregrtest/runtest_mp.py#L394-L418 The main thread tries to get a result from the output queue, but the queue is empty and remains empty. Although the queue.get() operation eventually times out (after 30 seconds), the main thread does not re-check if all the workers have exited (!), but instead retries the queue.get() in the "while True" loop. https://github.com/python/cpython/blob/8d7644fa64213207b8dc6f555cb8a02bfabeced2/Lib/test/libregrtest/runtest_mp.py#L415-L418 To reproduce, apply the below patch which introduces a small delay to more reliably trigger the hang. curl "https://gist.githubusercontent.com/colesbury/fe3769f43dfb724c86ecbb182b1f6749/raw/e29a4eaeebb8d5252cdd66f3f8a70f7bc5fa14e7/runtest_mp.diff"; | patch -p1 ./python -m test test_regrtest -m test_module_from_test_autotest -v ---------- messages: 409374 nosy: colesbury priority: normal severity: normal status: open title: Race condition in runtest_mp leads to hangs (never exits) versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker <[email protected]> <https://bugs.python.org/issue46205> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
