Daniel Sun created GROOVY-12402:
-----------------------------------
Summary: Local `gradlew test` oversubscribes small machines
Key: GROOVY-12402
URL: https://issues.apache.org/jira/browse/GROOVY-12402
Project: Groovy
Issue Type: Improvement
Reporter: Daniel Sun
h3. Problem
On a 6-CPU developer box, {{./gradlew test}} runs two Test tasks with
{{maxParallelForks = nproc/2}}, so six worker JVMs. Each is launched at
{{-Xmx2g}} with no {{-XX:ActiveProcessorCount}}, and G1/C2 then size themselves
for *all* six CPUs ({{ParallelGCThreads=6}}, {{CICompilerCount=3}} per JVM).
Load averages of 16-21 make a full suite take roughly twice as long as a less
contended run.
Three modules ({{performance}}, {{tests-preview}}, {{groovy-osgi-test}}) do not
apply {{org.apache.groovy-tested}}, so they sit outside
{{ConcurrentExecutionControlBuildService}} and inherit the ergonomic heap
(about one quarter of RAM). The indy parity tests then fork child JVMs with the
same default.
* 6 vCPU KVM guest, 23 GiB RAM, no swap
* JDK 25 (Corretto 25.0.4), Gradle 9.7.1
* {{master}} @ {{b142a89351}}, development environment ({{CI}} unset)
h3. Expected
Local test workers should not each assume they own every CPU. CI must stay
conservative: {{maxParallelForks=1}}, two concurrent Test tasks, two groovyc
forks.
h3. Before
{code}
./gradlew cleanTest test --no-build-cache
{code}
* Peak 6 Test Executor processes; {{-Xms512m -Xmx2g}}; no APC
* Full suite: 25m 41s; loadavg mean 15.0, max 21.7
* Five-module subset ({{performance}}, {{groovy-json}}, {{groovy-xml}},
{{groovy-templates}}, {{groovy-sql}}): 4m 11s; loadavg max 18.1
* 23731 passed, 131 skipped, 0 failed
h3. Approach
Keep the CI caps. Locally, size forks from the machine and tell each worker how
many CPUs it may use:
* {{maxParallelForks = max(1, (nproc-2)/2)}} (2 on six CPUs)
* {{-XX:ActiveProcessorCount = floor(nproc / peakWorkers)}} (1 on six CPUs)
* {{groovyJUnit_ms=256m}} ({{groovyJUnit_mx}} stays 2g)
* Enroll {{performance}}, {{tests-preview}}, and {{groovy-osgi-test}} in the
existing Test limiter; cap their heaps at 512m; pin parity-test children to
{{-Xmx256m -XX:ActiveProcessorCount=1}}
Knobs live on {{SharedConfiguration}} so Test and groovyc share one formula.
h3. After
Same command, same revision, same 23731 passed / 131 skipped:
||Run||Before||After||
|full cleanTest test --no-build-cache|25m 41s|22m 59s|
|five-module subset|4m 11s|2m 52s|
|peak Test JVMs|6|4|
|loadavg mean (full)|15.0|8.3|
|loadavg max (full)|21.7|15.1|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
