[issue38502] regrtest: use process groups

STINNER Victor added the comment: New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in branch '3.7': Update libregrtest from master (GH-19517) https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59 --

[issue38502] regrtest: use process groups

STINNER Victor added the comment: New changeset 67b8a1f0f0f78ec38b8626fa9f5b2f5a55c17e15 by Victor Stinner in branch '3.8': [3.8] Update libregrtest from master (GH-19516) https://github.com/python/cpython/commit/67b8a1f0f0f78ec38b8626fa9f5b2f5a55c17e15 --

[issue38502] regrtest: use process groups

STINNER Victor added the comment: The test_pty has been fixed. regrtest now calls setsid() when using multiprocessing (-jN), but only in the main branch. It's a nice feature, but I'm not comfortable about backporting it yet. Maybe if tests become too unstable on other branches, we may

[issue38502] regrtest: use process groups

STINNER Victor added the comment: > I don't know for sure that this is the cause but both 3.x builds following > this commit on my bolen-ubuntu worker (Ubuntu 18.04.3) have had test_pty > crash in the first attempt, with the retry succeeding. For example >

[issue38502] regrtest: use process groups

David Bolen added the comment: I can recreate this manually by running regrtest.py against test_pty. Crashes with any "-j#" option, but fine when run sequentially. Removing the process group change avoids the crash. With the process group change in place, the trigger point appears to be

[issue38502] regrtest: use process groups

David Bolen added the comment: I don't know for sure that this is the cause but both 3.x builds following this commit on my bolen-ubuntu worker (Ubuntu 18.04.3) have had test_pty crash in the first attempt, with the retry succeeding. For example

[issue38502] regrtest: use process groups

STINNER Victor added the comment: I will wait at least one day to see how buildbots like this change before backporting it to 3.7 and 3.8. -- ___ Python tracker ___

[issue38502] regrtest: use process groups

STINNER Victor added the comment: New changeset ecb035cd14c11521276343397151929a94018a22 by Victor Stinner in branch 'master': bpo-38502: regrtest uses process groups if available (GH-16829) https://github.com/python/cpython/commit/ecb035cd14c11521276343397151929a94018a22 --

[issue38502] regrtest: use process groups

STINNER Victor added the comment: See also bpo-12413 "make faulthandler dump traceback of child processes" which looks like a duplicate of bpo-18969. -- ___ Python tracker

[issue38502] regrtest: use process groups

STINNER Victor added the comment: See also bpo-18969: "test suite: enable faulthandler timeout in assert_python". This issue proposes to use killpg() to send a signal to all Python processes spawned by regrtest: trigger faulthandler to dump the Python traceback of all Python threads.

[issue38502] regrtest: use process groups

Change by STINNER Victor : -- keywords: +patch pull_requests: +16378 stage: -> patch review pull_request: https://github.com/python/cpython/pull/16829 ___ Python tracker ___

[issue38502] regrtest: use process groups

STINNER Victor added the comment: Example of the problem: sometimes, interrupting a multiprocessing test hangs. regrtest fails to interrupt a TestWorkerProcess thread. $ ./python -m test test_multiprocessing_fork -j10 -F --timeout=60 --slowest 0:00:00 load avg: 1.09 Run tests in parallel

[issue38502] regrtest: use process groups

New submission from STINNER Victor : Problem. When regrtest is interrupted by CTRL+c and regrtest multiprocessing code (-jN command line option) is used, regrtest randomly fails to stop come TestWorkerProcess threads. The problem is that only the direct child process is kill by SIGKILL. If