wombatu-kun commented on PR #19119: URL: https://github.com/apache/hudi/pull/19119#issuecomment-4862024523
Short answer: it slows only that one job's compile phase, not the parts that dominate CI time, and it is far cheaper than the OOM re-runs it prevents. Scope: the `-T 1` cap applies only to the `mvn clean install` reactor step of UT_FT_10, and only that job - every other job keeps the shared `-T 3`, and the three `mvn test` invocations (UT + FT) that make up the bulk of UT_FT_10's wall-clock are unchanged. The only thing that gets slower is that job's reactor compile/install, going 2 threads -> 1. I measured that step locally (full reactor `clean install -DskipTests`, `MAVEN_OPTS=-Xmx8g`, same `-Dscala-2.12 -Dspark3.5 -Dflink1.18` profiles as CI, warm `.m2`, 3 samples each; `thrift`/metaserver profiles dropped since I have no local thrift, which excludes only the small hudi-metaserver modules, not the bundle shade work). On a 12-core box: | Config | Samples (mm:ss) | Median | |--------|-----------------|--------| | `-T 2` (old) | 6:08, 5:20, 4:40 | 5:20 | | `-T 1` (new) | 7:15, 8:14, 7:52 | 7:52 | So the install step itself is ~2.5 min (~45%) slower. Two reasons that is not meaningful for total CI time: - It hits only the compile/install phase; the UT + FT test runs that dominate the job, and all other jobs, are unchanged. - It is dwarfed by the failure it prevents - the GCLocker OOM fails the whole job and forces a full re-run (recent hits #19117, #19004), repeating the entire compile + UT + FT sequence. These are local numbers on 12 cores; on a lower-core agent `-T 2`'s advantage should be smaller, so the real CI delta is likely under this. Net: a small, single-job compile slowdown in exchange for removing a recurring full-job failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
