dongjoon-hyun opened a new pull request, #974: URL: https://github.com/apache/spark-kubernetes-operator/pull/974
### What changes were proposed in this pull request? This PR aims to stop `AppValidateStep` from failing a `SparkApplication` with `spec.deploymentMode: ClientMode` again on every reconciliation. Like the Gateway API route validation, the `ClientMode` check now runs only while the application is `Submitted` or `ScheduledToRestart`, i.e., before the driver of an attempt is requested. Such an application still fails with `Client mode is not supported yet.` before any driver is requested. After that, it follows the usual path of a `Failed` application: `AppCleanUpStep` terminates it as `ResourceReleased` or schedules a restart according to `restartConfig`, and `SparkAppReconciler.cleanup()` removes the finalizer when it is deleted. ### Why are the changes needed? Since [0.7.0](https://github.com/apache/spark-kubernetes-operator/releases/tag/0.7.0) (2026-01-16), where SPARK-54674 made the `ClientMode` check effective, a `ClientMode` application falls into an infinite status-write loop and cannot be deleted. - `AppValidateStep` runs first in every reconciliation, and it checks `ClientMode` regardless of the current state. - For such an application, it always appends a new `Failed` state, persists it, and returns `completeAndImmediateRequeue()`. So `AppCleanUpStep` is never reached. - `SparkAppReconciler.cleanup()` also runs `AppValidateStep` first, so it always returns `DeleteControl.noFinalizerRemoval().rescheduleAfter(0)`. As a result, the application loops at the JOSDK rate limit (about 20 status updates per minute), and `status.stateTransitionHistory` grows without bound until the object approaches the etcd size limit. Deleting the application, or its namespace, never completes since the finalizer is never removed. This is the same pattern which SPARK-59720 fixed for `SparkCluster`. ### Does this PR introduce _any_ user-facing change? Yes, this fixes a bug. A `ClientMode` `SparkApplication` fails once, then terminates like any other failed application, e.g., `ttlAfterStopMillis` applies, and it can be deleted. The check no longer applies to an attempt whose driver is already requested, e.g., when `spec.deploymentMode` is changed to `ClientMode` while the application is running. It applies to the next attempt instead, like the Gateway API route validation. ### How was this patch tested? Pass the CIs with the newly added test cases, which fail without this fix. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
