dongjoon-hyun opened a new pull request, #58174: URL: https://github.com/apache/spark/pull/58174
### What changes were proposed in this pull request? This PR adds `private[spark] CoarseGrainedSchedulerBackend.supportsExecutorHold`, which tells whether a backend can hold its executors gracefully: publish an all-zero executor requirement to the cluster manager, and let the executors that are already running finish their tasks and exit on their own instead of being terminated. It is `false` by default, so a backend has to opt in: | Backend | Value | Reason | |---|---|---| | `CoarseGrainedSchedulerBackend` | `false` | `doRequestTotalExecutors` does not acknowledge the published requirement | | `StandaloneSchedulerBackend` | `true` | The Master honors a zero executor limit without killing the running executors | | `YarnSchedulerBackend` | `true` | The AM honors a zero executor target without killing the running executors | | `KubernetesClusterSchedulerBackend` | `true` only with `spark.kubernetes.allocation.pods.allocator=direct` | The other allocators scale their controller down on a zero requirement, deleting the running executor pods instead of letting them finish their tasks | ### Why are the changes needed? This is split out of https://github.com/apache/spark/pull/58054, which holds and resumes an application through graceful executor decommission. Whether a cluster manager keeps its running executors alive on a zero requirement is a per-resource-manager judgement, orthogonal to the rest of that PR and spanning Standalone, YARN, and Kubernetes, so it is reviewed on its own. The method has no caller until #58054 lands. ### Does this PR introduce _any_ user-facing change? No because this is a new `private[spark]` method. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
