> Based on data from pervious runs, this setting optimizes the runorder > of the tests to minimize total run-time.
This is a digression, but I think may be inspirational. Kristian's description matches how I implemented per-jvm balancing in our JUnit4 runner initially and I have a suggestion on how to make it better. In practice a similar near-optimal balancing can be achieved by doing initial balancing of, say, 50% of all suites, followed by serving any of the remaining test suites to any idle slave JVM (much like job stealing in fork-join). While it may happen that the longest test will be scheduled last, it is unlikely in practice (given a lot of tests and their permutations). Also, if history of runs is available, the above behavior will still work and even provide a shield against tests that have a large variance on execution time (when static scheduling may be incorrect). We did implement a Maven plugin for running JUnit (4.x) tests that complements (or replaces, if one wishes to do so) Surefire so you can compare both approaches. The code to do the above trick does add some complexity because it requires master-slave communication to take place, but this is relatively easy to do and provides other benefits (and we needed it anyway to provide consistent reporting over slave JVMs). Grab anything you want, the project is in our labs and it is Apache licensed: http://labs.carrotsearch.com/randomizedtesting.html Load balancing is just part of what the project is about; the code I mentioned above is mostly around here: https://github.com/carrotsearch/randomizedtesting/blob/master/integration-ant/ant-junit4/src/main/java/com/carrotsearch/ant/tasks/junit4/JUnit4.java#L472 Maven integration can be seen as an integration test (with scarce documentation yet) here: https://github.com/carrotsearch/randomizedtesting/blob/master/integration-maven/junit4-maven-plugin-tests/src/it/01-basic-test/pom.xml I've used it in another project, so a cleaner example of use from within a POM is here (you should disable surefire or your tests will run twice): https://github.com/carrotsearch/hppc/blob/master/hppc-core/pom.xml#L217 Dawid > > Given for tests: > > A 7 minutes > B 5 minutes > C 1 minute > D 1 minute > > Previously you'd be *required* to have 4 threads to get the minimum > guaranteed run-time, which is 7 minutes. > Both 2 and 3 threads had a worst-case run-time behaviour of 12 minutes. > > With surefire 2.11 you'd be guaranteed 7 minutes total run-time for > threads=2 (parallel=classes). > For 2.12 you get the same guarantee with forkMode=perthread and > threadCount=2. > > Hope that makes it clear ;) > > Kristian > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
