[
https://issues.apache.org/jira/browse/GROOVY-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul King resolved GROOVY-12402.
--------------------------------
Fix Version/s: 7.0.0-alpha-1
Assignee: Daniel Sun
Resolution: Fixed
> Local `gradlew test` oversubscribes small machines
> --------------------------------------------------
>
> Key: GROOVY-12402
> URL: https://issues.apache.org/jira/browse/GROOVY-12402
> Project: Groovy
> Issue Type: Improvement
> Reporter: Daniel Sun
> Assignee: Daniel Sun
> Priority: Major
> Fix For: 7.0.0-alpha-1
>
>
> h3. Problem
> On a 6-CPU developer box, {{./gradlew test}} runs two Test tasks with
> {{maxParallelForks = nproc/2}}, so six worker JVMs. Each is launched at
> {{-Xmx2g}} with no {{-XX:ActiveProcessorCount}}, and G1/C2 then size
> themselves for *all* six CPUs ({{ParallelGCThreads=6}}, {{CICompilerCount=3}}
> per JVM). Load averages of 16-21 make a full suite take roughly twice as long
> as a less contended run.
> Three modules ({{performance}}, {{tests-preview}}, {{groovy-osgi-test}}) do
> not apply {{org.apache.groovy-tested}}, so they sit outside
> {{ConcurrentExecutionControlBuildService}} and inherit the ergonomic heap
> (about one quarter of RAM). The indy parity tests then fork child JVMs with
> the same default.
> * 6 vCPU KVM guest, 23 GiB RAM, no swap
> * JDK 25 (Corretto 25.0.4), Gradle 9.7.1
> * {{master}} @ {{b142a89351}}, development environment ({{CI}} unset)
> h3. Expected
> Local test workers should not each assume they own every CPU. CI must stay
> conservative: {{maxParallelForks=1}}, two concurrent Test tasks, two groovyc
> forks.
> h3. Before
> {code}
> ./gradlew cleanTest test --no-build-cache
> {code}
> * Peak 6 Test Executor processes; {{-Xms512m -Xmx2g}}; no APC
> * Full suite: 25m 41s; loadavg mean 15.0, max 21.7
> * Five-module subset ({{performance}}, {{groovy-json}}, {{groovy-xml}},
> {{groovy-templates}}, {{groovy-sql}}): 4m 11s; loadavg max 18.1
> * 23731 passed, 131 skipped, 0 failed
> h3. Approach
> Keep the CI caps. Locally, size forks from the machine and tell each worker
> how many CPUs it may use:
> * {{maxParallelForks = max(1, (nproc-2)/2)}} (2 on six CPUs)
> * {{-XX:ActiveProcessorCount = floor(nproc / peakWorkers)}} (1 on six CPUs)
> * {{groovyJUnit_ms=256m}} ({{groovyJUnit_mx}} stays 2g)
> * Enroll {{performance}}, {{tests-preview}}, and {{groovy-osgi-test}} in the
> existing Test limiter; cap their heaps at 512m; pin parity-test children to
> {{-Xmx256m -XX:ActiveProcessorCount=1}}
> Knobs live on {{SharedConfiguration}} so Test and groovyc share one formula.
> h3. After
> Same command, same revision, same 23731 passed / 131 skipped:
> ||Run||Before||After||
> |full cleanTest test --no-build-cache|25m 41s|22m 59s|
> |five-module subset|4m 11s|2m 52s|
> |peak Test JVMs|6|4|
> |loadavg mean (full)|15.0|8.3|
> |loadavg max (full)|21.7|15.1|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
