In 2008 Alex Vandiver contributed a patch to "prove" that allowed you to specify that you wanted some tests to run in parallel and others in serial. This is a great feature for those of us with large test suites that would like to test advantage of parallism, but have suites that aren't 100% parallel-ready. I'm grateful that work was done.
The feature was considered experimental, and was not documented in `prove`. From what I can tell it has remained largely undiscovered, and is still remains in the same state about 4 years later. In the last few days I started working on the same problem myself, not realizing that the undocumented feature exist. I asked on StackOverflow [1], and proceeded to code up my own solution [2] before I ended up here to report my results and ask for help. 1. http://stackoverflow.com/questions/11977015/how-to-run-some-but-not-all-tests-in-a-perl-test-suite-in-parallel/11977495#11977495 2. https://github.com/Perl-Toolchain-Gang/Test-Harness/pull/3 After I fixed in a bug in calculating the tests to run in my own solution, it appeared to work, but then at the end of the run, no summary would be reported. It would just stop and it wasn't clear what the problem was. So, I tried to see if I could get the "--rules" option to work for me. First I tried putting this syntax in my my .proverc: --rules seq=t/first_serial_test.t --rules seq=t/second_serial_test.t It appeared that it was starting to run my serial tests first... but it appeared to keep going with running everything in serial, despite "-j 4" on the command line. So, I tried this: --rules seq=t/first_serial_test.t --rules seq=t/second_serial_test.t --rules par=** This appeared to have the opposite result. No *everything* appeared to be run in parallel. Finally, I tried this variation: --rules par=** --rules seq=t/first_serial_test.t --rules seq=t/second_serial_test.t This appeared that it was doing the right thing.... until it failed the same way own patch did.... the test suite run just ended, with no summary. Perhaps is this simply a documentation issue, and I haven't gotten the syntax just right. How do I specify that there are some tests that I always want to be run in serial? My goal is to be able to specify this list of exceptions once in a file and forget about it. I don't want the existence of a rule to imply that I want to that test. I only want rules applied if tests are actually selected to run. So far, I'm not sure if specify a rule also implies that I'm selecting a test to run, which I wouldn't want. Thanks for considering this feature again with me. Let's get it finished, documented and published! *UPDATE* After I drafted this message, I found Test::Steering, which also advertises the feature of mixing parallel and serial runs. However, I found it didn't work. First, I had to patch it just to get basic functionality going [3], and then when I got a real test to run, it produced two "summary" reports... the one for the parallel runs was printed in the middle of the run, while a second summary just for the serial runs appeared at the end. https://rt.cpan.org/Ticket/Display.html?id=62681 https://metacpan.org/module/Test::Steering Perhaps everyone in this problem space using the "Roll your own" approach, but it sure seems like there's potential for a generally useful, re-usable tool for this. Thanks! Mark
