apupier commented on PR #28303: URL: https://github.com/apache/camel/pull/28303#issuecomment-6748945503
Benchmark of `camel-spring-xml` (1186 tests, `mvn surefire:test`, offline, 22-core / 30 GB Linux machine). The "4 vCPU" column is the same machine pinned with `taskset -c 0-3` to simulate a GitHub `ubuntu-latest` runner. The simulation matches the real CI: 4 forks ≈ 2'19" here vs 2'24" on CI, 2 forks ≈ 3'40" here vs 4'33" on the CI of this PR. | Config | 4 vCPU (CI sim) | 22 cores | |---|---|---| | current: 4 forks, sequential | 139 s | 128 s | | `0.5C` on 4 vCPU = 2 forks, sequential | 220 s | – | | `0.5C` on 22 cores = 11 forks, sequential | – | **90 s** | | `1C` on 22 cores = 22 forks, sequential | – | 378 s | | 1 fork, JUnit parallel classes, factor 1 | 147–157 s | 109–110 s | | 2 forks, parallel, factor 1 | 225 s | 208 s | | 4 forks, parallel, factor 0.5 | 137 s | – | | 4 forks, parallel, factor 1 | – | 125 s | Takeaways: - This confirms the review comments: `0.5C` goes from 4 to 2 forks on a 4 vCPU runner and is ~60% slower there. - `1C` is fine on 4 vCPU (= 4 forks, same as today), but on 22 cores it is ~4x slower than 11 forks (98 test classes took more than 10 s vs 5), so it is not a good value for big machines either. This is CPU/memory contention, no OutOfMemory. - Running test classes in parallel inside fewer forks does not help on 4 vCPU (the wall time is bounded by CPU and a few slow classes), but one parallel fork is close to the best local result (~110 s vs 90 s) with less memory. - Running parallel classes was not possible before: with 1 fork it gave ~115 failures because of shared state (H2 database name, `testDirectory` system property, platform MBeanServer). I fixed those in #29515 (tests only, default build unchanged), after which 1 fork + parallel passes all 1186 tests. My suggestion for this PR: keep `4` (or use `1C` if we want it to scale up) and leave local speed-ups to `-Dcamel.surefire.forkCount=0.5C`. Jenkins still needs to be validated as you said. _IBM Bob 2.2.1 on behalf of @apupier. This comment was AI-generated; please verify._ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
