dongjoon-hyun opened a new pull request, #58618: URL: https://github.com/apache/spark/pull/58618
### What changes were proposed in this pull request? This PR proposes to turn recovery mode off when a held application resumes, so that it allocates normal executors again. - `ExecutorPodsAllocator.unsetRecoveryMode()` removes `spark.kubernetes.allocation.recoveryMode.enabled` from the driver conf if the driver turned it on after an OOM. A value configured by the user is left untouched. - `KubernetesClusterSchedulerBackend.setExecutorsHeld(false)` calls it, which is the hook `SparkContext.resumeExecutors()` uses. - `running-on-kubernetes.md` is updated accordingly. ### Why are the changes needed? Recovery mode stays on for the rest of the driver's lifetime once an OOM is detected. A hold drains every executor, including the recovery-mode ones, and a resume starts allocating from scratch, so it is the natural point to start over with the configured executor cores. Otherwise a long-running driver, e.g. a Spark Connect server, stays on single-task executors after a single OOM until it is restarted. ### Does this PR introduce _any_ user-facing change? No for the released versions, since hold and resume (4.4.0) is not released yet. Within the unreleased branches, a K8s application that entered recovery mode after an OOM allocates normal executors again after a hold and resume. An explicitly configured `spark.kubernetes.allocation.recoveryMode.enabled=true` stays on as before. ### How was this patch tested? Pass the CIs with the newly added test cases. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
