cloud-fan opened a new pull request, #56631: URL: https://github.com/apache/spark/pull/56631
### What changes were proposed in this pull request? Followup to https://github.com/apache/spark/pull/55738. The three `INJECT_SHUFFLE_FETCH_FAILURES` injection-state maps in `DAGScheduler` (`injectShuffleFetchFailuresCorruptedAttempt`, `injectShuffleFetchFailuresPendingDelayedCorruption`, `injectShuffleFetchFailuresDownstreamSuccessCount`) are initialized with `if (Utils.isTesting) new ConcurrentHashMap else null`. This patch allocates them unconditionally instead. ### Why are the changes needed? The cleanup site in `cleanupStateForJobAndIndependentStages` (and the other use-sites) re-evaluate `if (Utils.isTesting)` to decide whether to dereference these maps. `Utils.isTesting` reads the mutable `spark.testing` system property, so it can return a different value when the `DAGScheduler` is constructed than at the later use-sites. If it does, a `null` map is dereferenced and the `DAGScheduler` event loop crashes with a `NullPointerException`. Allocating the maps unconditionally makes the construction guard and the use-site guards consistent, so they can never disagree. ### Does this PR introduce _any_ user-facing change? No. The maps are only ever populated inside the config-gated `INJECT_SHUFFLE_FETCH_FAILURES` test paths, so in production they stay empty and there is no behavioral change beyond allocating an empty map. ### How was this patch tested? Existing tests. This is a defensive initialization change; the maps stay empty unless the existing `INJECT_SHUFFLE_FETCH_FAILURES` test paths populate them, so the existing coverage exercises them unchanged. ### Was this patch authored or co-authored using generative AI tooling? Yes, drafted with Claude Code (Anthropic). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
