> Recently, I started to experiment "./python -m test [options] -F -j100" to attempt to reproduce some tricky race conditions: -j100 spawns 100 worker processes in parallel and -F stands for --forever (run tests in loop and stop at the first failure).
Interesting, even as someone who has recently worked within asyncio a decent amount recently, I was entirely unaware of this method of producing race conditions in the tests. If continued experimentation shows this to be a reliable means of reproducing various race conditions, it might be worth adding this to https://devguide.python.org/runtests/. Thanks for sharing! On Mon, Oct 21, 2019 at 7:08 AM Victor Stinner <vstin...@python.org> wrote: > Hi, > > Right now, there are 14 open issues with "test_asyncio" in the title. > Many test_asyncio tests have race conditions. I'm trying to fix them > one by one, but it takes time, and then new tests are added with new > race condition :-( For example, the following new test is failing > randomly on Windows: > > "Windows: test_asyncio: test_huge_content_recvinto() fails randomly > with ProactorEventLoop" is failing randomly since 6 months: > https://bugs.python.org/issue36732 > > test_asyncio uses more and more functional tests which is a good > thing. In the early days of asyncio, most tests mocked more than half > of asyncio to really be "unit test". But at the end, the test tested > more mocks than asyncio... The problem of functional tests is that > it's hard to design them properly to avoid all race conditions, > especially when you consider multiplatform (Windows, macOS, Linux, > FreeBSD, etc.). > > It would help me if someone could try to investigate these issues, > provide a reliable way to reproduce them, and propose a fix. (Simply > saying that you can reproduce the test and that you would like to work > on an issue doesn't really help, sorry.) > > Recently, I started to experiment "./python -m test [options] -F > -j100" to attempt to reproduce some tricky race conditions: -j100 > spawns 100 worker processes in parallel and -F stands for --forever > (run tests in loop and stop at the first failure). I was surprised > that my Fedora 30 didn't burn in blame. In fact, the GNOME desktop > remains responsible even with a system load higher than 100. The Linux > kernel (5.2) is impressive! Under such high system load (my laptop has > 8 logical CPUs), race conditions are way more likely. > > The problem of test_asyncio is that it's made of 2160 tests, see: > > ./python -m test test_asyncio --list-cases > > You may want to only run a single test case (class) or even a single > test method: see --match option which can be used multiple times to > only run selected test classes or selected test methods. See also > --matchfile which is similar but uses a file. Example: > > $ ./python -m test test_asyncio --list-cases > cases > # edit cases > $ ./python -m test test_asyncio --matchfile=cases > > test_asyncio is one of the most unstable test: I'm getting more and > more buildbot-status emails about test_asyncio... likely because we > fixed most of the other race conditions which is a good thing ;-) > > Some issues look to be specific to Windows, but it should be possible > to reproduce most issues on Linux as Linux. Sometimes, it's just that > some specific Windows buildbot workers are slower than other buildbot > workers. > > Good luck ;-) > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/R7X6NKGEOKWD3PBWIL2LPZWZ6MMRANN5/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7K4Z3QJTZCRALCIIFEJSQXVCMBYU37J2/ Code of Conduct: http://python.org/psf/codeofconduct/
