[issue25285] regrtest: run tests in subprocesses with -j1 on buildbots

2016-05-20 Thread STINNER Victor

STINNER Victor added the comment:

changeset:   101449:6f7e5b51c8fb
user:Victor Stinner 
date:Fri May 20 13:15:55 2016 +0200
files:   Lib/test/libregrtest/cmdline.py Misc/NEWS Tools/buildbot/test.bat
description:
regrtest doesn't ignore -j1 anymore

* regrtest now uses subprocesses when the -j1 command line option
  is used: each test file runs in a fresh child process. Before, the -j1 option
  was ignored.
* Tools/buildbot/test.bat script now uses -j1 by default to run
  each test file in fresh child process.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25285] regrtest: run tests in subprocesses with -j1 on buildbots

2015-10-12 Thread Arfrever Frehtes Taifersar Arahesis

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25285] regrtest: run tests in subprocesses with -j1 on buildbots

2015-09-30 Thread STINNER Victor

New submission from STINNER Victor:

Currently, regrtest ignores the -j1 parameter. For -jN, the multiprocess mode 
is only enabled for N != 1. I propose to modify regrtest to run tests in 
subprocesses when -j1 is used. It would solve the issue #18906.

I also want to run tests in subprocesses in buildbot to better isolate tests. 
Running all tests in the same process, sequentially, cause various issues:

* various kinds of side effects
* memory leaks
* random failures (unreliable tests)

"make buildbot" already runs regrtest with -j1. I propose to also modify 
Tools/buildbot/test.bat to use run tests with -j1 by default on Windows.

I fixed regrtest to setup tests using the same code for classic mode and 
multiprocess mode: slaves (child processes running tests for regrtest -jN) now 
inherit
--memlimit/-M, --threshold/-t and --nowindows/-n options.

Only 3 remaining functions are incompatible with -jN: --coverage/-T and 
--findleaks/-l.

Finally, the multiprocess mode of regrtest now displays the duration of tests 
which took longer than 30 seconds, and every minute, it displays the tests 
running since longer than 30 seconds. Example with multiple workers (-j4):
---
...
[395/399/1] test_mimetypes -- running: test_lzma (111 sec), 
test_multiprocessing_forkserver (224 sec)
[396/399/1] test_poplib -- running: test_lzma (111 sec), 
test_multiprocessing_forkserver (225 sec)
[397/399/1] test_lzma (111 sec) -- running: test_multiprocessing_forkserver 
(225 sec)
[398/399/1] test_range -- running: test_multiprocessing_forkserver (227 sec)
running: test_multiprocessing_forkserver (287 sec)
[399/399/1] test_multiprocessing_forkserver (340 sec)
378 tests OK.
---

Note: "python -m test -j1 -j4" uses 4 workers, only the last -jN parameter is 
used. Some buildbots add -jN to use more workers.

--
components: Tests
messages: 251962
nosy: haypo
priority: normal
severity: normal
status: open
title: regrtest: run tests in subprocesses with -j1 on buildbots
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25285] regrtest: run tests in subprocesses with -j1 on buildbots

2015-09-30 Thread STINNER Victor

Changes by STINNER Victor :


--
keywords: +patch
Added file: http://bugs.python.org/file40632/regrtest_run_in_subprocess.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com