Hi there I'm really enjoying using pytest (and currently porting a large OSS project <https://github.com/OSGeo/gdal/pull/963> to it).
The project I'm porting has a large number of tests that depend on other tests. In particular, combining certain tests in the same run as other tests appears to cause segfaults sometimes, or double-free errors at process exit (particularly annoying since it's very hard to pinpoint what caused it) My current workflow for finding these dependencies involves: * run all the tests, note a failure. * run half the tests, see if it still fails. If not, run the other half * run half of *those *tests, etc. This is pretty similar to the `git bisect` workflow when debugging when a bug was introduced in a codebase. I'd like to find or write a similar plugin for pytest, that would repeatedly run teh tests with smaller and smaller subsets of tests until the dependent tests are identified. Since sometimes the failure mode I'm tracking is a crash at the end of the process (after all tests have succeeded), I will need to run tests as a subprocess of the main process (so I can restart it when it fails.) The simplest implementation I can think of is a plugin that completely takes over the parent pytest process, runs pytest itself repeatedly in a `subprocess.check_call()`, then calls `sys.exit(0)` when done. This sounds pretty wrong but it could be a good starting point? However, I note the excellent pytest-xdist plugin is good at spawning worker subprocesses and sending tests to them, and rather than re-invent the wheel I wondered if I could make my plugin depend on xdist, and *control* the xdist workers (e.g. run one worker with all the tests; if it crashes, run a new worker with half the tests, etc). Could someone knowledgeable in this area give me some pointers on whether this is a good idea; what exactly is possible; and where to look for code to re-use? Particularly if you know of any plugins that might do similar things that I could take inspiration from. Thanks for reading :) Craig de Stigter
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
