> gradlew -p lucene test
> That took my machine 27 minutes! I can see it used like 7 threads when I'd
> rather it use 3-4. That's probably why.
I capped the execution of parallel tests to this
(gradle/testing/defaults-tests.gradle):
// Set up default parallel execution limits.
maxParallelForks = (int) Math.max(1,
Math.min(Runtime.runtime.availableProcessors() / 2.0, 3.0))
but this limit applies per-project and gradle runs with N workers in
parallel mode (where N equals the number of cores). For example, on my
8-core Linux machine (SSD drive) the test time for
./gradlew -p lucene cleanTest test -Ptests.seed=28BC12BA7C2CD894
is 11m 32s. If I limit max workers to 4:
./gradlew -p lucene cleanTest test -Ptests.seed=28BC12BA7C2CD894 --max-workers=4
the build time is XXX.
It is obvious we need to fine-tune this but it's not obvious what the
defaults should be because there are many dimensions to performance:
I/O congestion, CPU cores, overall memory bandwidth and operating
system (underlying filesystem implementation mostly, I believe). Also,
as Mark pointed out before, the test runner is slightly different in
Gradle and doesn't load-balance tests too efficiently (no work
stealing).
Finally, all the above also doesn't mean we can only think of
improving gradle performance and not tests themselves... Some of them
are just (very) slow. :)
If you're willing to experiment then try to run with a varying number
of workers (and identical seed to keep the tests the same). Then see
which one turns out to be the best for you (and report back). I think
it'll be half the number of cores (effectively physical cores) unless
you have a very beefy machine when memory bandwidth and I/O comes into
play.
gradlew --max-workers N -Ptests.seed=deadbeef
> gradlew :helpAnt
> This failed: -- java.io.FileNotFoundException
Corrected, apologies.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
