HyukjinKwon opened a new pull request, #56785: URL: https://github.com/apache/spark/pull/56785
### What changes were proposed in this pull request? Follow-up to [SPARK-57650](https://issues.apache.org/jira/browse/SPARK-57650), which fixed the deterministic "AM stuck in ACCEPTED" hang in `BaseYarnClusterSuite`. Two further **test-only** changes to reduce the remaining flakiness of `YarnClusterSuite`: 1. Give the single mini `NodeManager` 8GB (`yarn.nodemanager.resource.memory-mb` + `yarn.scheduler.maximum-allocation-mb`) so executor allocation is never starved once the ~1.4GB AM is running. 2. Raise the executor→driver connection retry budget for the launched apps (`spark.rpc.io.maxRetries=10`, `spark.rpc.io.retryWait=2s`) so a transient RPC-accept stall does not permanently fail an executor. These are defaults that individual tests can still override via `extraConf`. ### Why are the changes needed? Even after SPARK-57650, the scheduled `Build / Java21` and `Build / Java25` master lanes fail in the `yarn` module roughly **50% of runs** (e.g. fork run `28151220075` PASS vs `28151247521` FAIL — same commit, 40s apart). All failures are the same six `YarnClusterSuite` tests timing out after 3 minutes (`The code passed to eventually never returned normally ... handle.getState().isFinal() was false`). From the `yarn-app-log` / `unit-tests-log` artifacts, the AM/driver comes up, but the executor (and sometimes the AM) intermittently fail to connect back to the driver's RPC server on `localhost` (`java.io.IOException: Failed to connect to localhost/127.0.0.1:<port>`, connection refused). The in-JVM mini RM+NM, the driver subprocess and the AM/executor JVMs all contend for CPU on a single CI runner, so the driver's accept loop occasionally stalls; an executor that loses this race exits after the default 3 connection retries, and the application can then never reach a final state. ### Does this PR introduce any user-facing change? No. Test-only. ### How was this patch tested? `YarnClusterSuite` was previously failing ~50% of the time. With this change the `yarn` module job was run **6 times** on the fork; all 6 passed, with `YarnClusterSuite` reporting `tests=30, failures=0, skipped=0` (the 6 formerly-failing tests now pass): - Before (master, failing): apache/spark run `28148781009` (`Build / Java21`) — 6 `YarnClusterSuite` timeouts. - After (this branch): HyukjinKwon/spark runs `28162182834`, `28162247111`, `28162249819`, `28175759262`, `28175762257`, `28175765871` — `yarn` job green in all six. ### Was this patch authored or co-authored using generative AI tooling? Yes, Generated-by: Claude Code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
