We don't have the two-way comms channel between the plugin and the fork in surefire which means it's all precalculated. There are certainly benefits/drawbacks to both approaches. My personal use-case involves extremely long-running tests and of extreme variation in run-time, which seems to fit better with the implementation in surefire at the moment.
The code to make the comms channel 2 way is already written, and making reusable forks is one of the few remaining things I'd like to add to surefire. But I'm not sure I'm going to be prioritizing that much further, there's so many other things that need to be done ;) It's a fairly simple patch to write up, just needs a lot of tests ;) Kristian 2012/1/29 Dawid Weiss <[email protected]>: >> Based on data from pervious runs, this setting optimizes the runorder >> of the tests to minimize total run-time. > > This is a digression, but I think may be inspirational. > > Kristian's description matches how I implemented per-jvm balancing in > our JUnit4 runner initially and I have a suggestion on how to make it > better. In practice a similar near-optimal balancing can be achieved > by doing initial balancing of, say, 50% of all suites, followed by > serving any of the remaining test suites to any idle slave JVM (much > like job stealing in fork-join). While it may happen that the longest > test will be scheduled last, it is unlikely in practice (given a lot > of tests and their permutations). Also, if history of runs is > available, the above behavior will still work and even provide a > shield against tests that have a large variance on execution time > (when static scheduling may be incorrect). > > We did implement a Maven plugin for running JUnit (4.x) tests that > complements (or replaces, if one wishes to do so) Surefire so you can > compare both approaches. The code to do the above trick does add some > complexity because it requires master-slave communication to take > place, but this is relatively easy to do and provides other benefits > (and we needed it anyway to provide consistent reporting over slave > JVMs). Grab anything you want, the project is in our labs and it is > Apache licensed: > > http://labs.carrotsearch.com/randomizedtesting.html > > Load balancing is just part of what the project is about; the code I > mentioned above is mostly around here: > https://github.com/carrotsearch/randomizedtesting/blob/master/integration-ant/ant-junit4/src/main/java/com/carrotsearch/ant/tasks/junit4/JUnit4.java#L472 > > Maven integration can be seen as an integration test (with scarce > documentation yet) here: > https://github.com/carrotsearch/randomizedtesting/blob/master/integration-maven/junit4-maven-plugin-tests/src/it/01-basic-test/pom.xml > > I've used it in another project, so a cleaner example of use from > within a POM is here (you should disable surefire or your tests will > run twice): > https://github.com/carrotsearch/hppc/blob/master/hppc-core/pom.xml#L217 > > Dawid > >> >> Given for tests: >> >> A 7 minutes >> B 5 minutes >> C 1 minute >> D 1 minute >> >> Previously you'd be *required* to have 4 threads to get the minimum >> guaranteed run-time, which is 7 minutes. >> Both 2 and 3 threads had a worst-case run-time behaviour of 12 minutes. >> >> With surefire 2.11 you'd be guaranteed 7 minutes total run-time for >> threads=2 (parallel=classes). >> For 2.12 you get the same guarantee with forkMode=perthread and >> threadCount=2. >> >> Hope that makes it clear ;) >> >> Kristian >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
