On Wed, 13 Sep 2017 15:42:56 +0200 Victor Stinner <victor.stin...@gmail.com> wrote: > I would like to see if and how we can integrate/move some regrtest > features into the unittest module. Example of regrtest features: > > * skip a test if it allocates too much memory, command line argument > to specify how many memory a test is allowed to allocate (ex: > --memlimit=2G for 2 GB of memory)
That would be suitable for a plugin if unittest had a plugin architecture, but not as a core functionality IMO. > * concept of "resource" like "network" (connect to external network > servers, to the Internet), "cpu" (CPU intensive tests), etc. Tests are > skipped by default and enabled by the -u command line option (ex: "-u > cpu). Good as a core functionality IMO. > * track memory leaks: check the reference counter, check the number of > allocated memory blocks, check the number of open file descriptors. Good for a plugin IMO. > * detect if the test spawned a thread or process and the > thread/process is still running at the test exit Good for a plugin IMO. > * --timeout: watchdog killing the test if the run time exceed the > timeout in seconds (use faulthandler.dump_traceback_later) Good for a plugin IMO. > * multiprocessing: run tests in subprocesses, in parallel Good as a core functionality IMO. > * redirect stdout/stderr to pipes (StringIO objects), ignore them on > success, or dump them to stdout/stderr on test failure Good for a plugin IMO. > * --slowest: top 10 of the slowest tests Good for a plugin IMO. > * --randomize: randomize test order Will be tricky to mix with setupClass. > * --match, --matchfile, -x: filter tests Good as a core functionality IMO. > * --forever: run the test in a loop until it fails (or is interrupted by > CTRL+c) Good for a plugin IMO. > * --list-tests / --list-cases: list test files / test methods Good as a core functionality IMO. > * --fail-env-changed: mark tests as failed if a test altered the environment Good for a plugin IMO. > * detect if a "global variable" of the standard library was modified > but not restored by the test: Good for a plugin IMO. > * test.bisect: bisection to identify the failing method, used to track > memory leaks or identify a test leaking a resource (ex: create a file > but don't remove it) Good as a core functionality IMO. > I started to duplicate code in many files of Lib/test/test_*.py to > check if tests "leak running threads" ("dangling threads"). Example > from Lib/test/test_theading.py: > > class BaseTestCase(unittest.TestCase): > def setUp(self): > self._threads = test.support.threading_setup() > > def tearDown(self): > test.support.threading_cleanup(*self._threads) > test.support.reap_children() > > I would like to get this test "for free" directly from the regular > unittest.TestCase class, but I don't know how to extend the unittest > module for that? Instead of creating tons of distinct base TestCase classes, you can just provide helper functions / methods calling addCleanup(). Regards Antoine. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/