I filed this issue so that the test execution statistics from a local machine could be reused:
https://issues.apache.org/jira/browse/LUCENE-4654 Dawid On Thu, Jan 3, 2013 at 8:45 AM, Dawid Weiss <[email protected]> wrote: >> Is this how it works, and if so, > > Yes and no. > > First, test classes are scattered pseudo-randomly (depending on the > master seed) across forked JVMs. This is as to ensure repeatability in > case of test-suite-interdependencies (like one class leaving something > in a static field and another class depending on it) this order needs > to be preserved between runs. > > Second, the allocation of test classes to forked JVMs is not done at > random; there is a greedy heuristic that tries to level the load on > all forked JVMs. These statistics, however, are computed and cached in > advance so they'll be more and more off target as time passes, > depending on the different (random) configuration of selected > components, nightly/ weekly switches etc, particular hardware > configuration, etc. You could try to recompute up-to-date stats for > your machine: > > # Update global (versioned!) execution times cache (top level). > ant clean test > ant -f lucene/build.xml test-updatecache > > and then the balancing should improve. Read on. > >> is it a major undertaking to have it assign >> tests to JVMs as each suite is completed, rather than at the beginning? > > This feature is actually already in the code, but (for reasons above) > not active by default. If you want you can run in job-stealing mode > where suite classes are dispatched to forked JVMs as they become > available. When you run 'ant test-help' from top-level Solr/Lucene > checkout there is this hint which is of relevance: > > # Use repeatable suite order on slave JVMs (disables job stealing). > ant -Dtests.dynamicAssignmentRatio=0 test > > this means "fully preallocated assignment". To have exactly the > opposite (no preallocated assignment) you'd do: > > ant -Dtests.dynamicAssignmentRatio=1 test > > Anything in between is partially preallocated queue with a tail > allocated on demand. Knowing this, if you inspect Lucene build file > sources you'll notice that the value of this parameter is: > > <property name="tests.dynamicAssignmentRatio" value="0.50" /> > <!-- 50% of suites --> > > So what I said at the beginning is not true -- we actually preallocate > only 50% of the suite classes to forked JVMs, the rest is dispatched > on demand. Like I said though, the allocation stats may be so off > target that in your case this is not enough. Just set the assignment > ratio to '1'. Note that even then the last JVM may be given a long > suite so the execution time won't be "even". > > Final note. If you're an algorithmic freak and think the allocation > model should use an optimal heuristic (greedy allocation of large > tasks first) then I can tell you in advance that this is a wrong > approach here. The first reason is that we want to have a uniform > sample from the set of permutations of all test suites running on > forked JVMs so that we can detect suite-dependencies early. The second > reason is more mundane -- > in practice if we feed 4 large, heavy test suites to forked JVMs at > the same time they start to compete for resources (i/o, memory bus) > and in the end the execution lasts longer compared to the random > version. > > Dawid --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
