dongjoon-hyun opened a new pull request, #885: URL: https://github.com/apache/spark-kubernetes-operator/pull/885
### What changes were proposed in this pull request? This PR aims to release the Kueue `Workload` of `SparkApplication` only after its driver and executor pods are gone, like `ClusterSuspendStep` does for `SparkCluster`. - `AppCleanUpStep` deletes the driver resources first, keeps the application in its current state until no driver or executor pod remains, and then deletes the `Workload`. It still does so before a restart is scheduled. - Only an application that holds Kueue quota waits. The wait is bounded by `forceTerminationGracePeriodMillis` from the start of the cleanup: the attempt stop, the deletion, or the retention expiry. - `ReconcilerUtils.hasPods` extracts the remaining-pod check of `ClusterSuspendStep` so that `SparkAppContext` can reuse it. ### Why are the changes needed? This is a follow-up of the following review comment. - https://github.com/apache/spark-kubernetes-operator/pull/879#discussion_r4091942897 Previously, `AppCleanUpStep` released the `Workload` before it deleted the driver pod. So Kueue could admit another workload while the driver and executors were still terminating, and briefly overcommit the `ClusterQueue`. ### Does this PR introduce _any_ user-facing change? Yes. A `SparkApplication` with a Kueue `Workload` now stays in its stopping state until its pods are gone. There is no change without the Kueue integration, which is disabled by default. ### How was this patch tested? Pass the CIs with the newly added test cases in `AppCleanUpStepTest`, `SparkAppReconcilerTest`, and `SparkAppContextTest`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
