dongjoon-hyun opened a new pull request, #857: URL: https://github.com/apache/spark-kubernetes-operator/pull/857
### What changes were proposed in this pull request? This PR stops `AppInitStep` and `ClusterInitStep` from taking a failed read of the driver pod or the master `StatefulSet` for a resource that was never requested. `SparkAppContext.getCurrentAttemptDriverPod` propagates a failed verification instead of reporting the pod as absent, and `ClusterInitStep.isMasterRequested` reads the `StatefulSet` directly rather than through the lenient `ReconcilerUtils.getResource`. Both steps now requeue on such a failure: the suspend branch, moved into a `holdForSuspend` method, neither releases the Kueue Workload nor requests the resources, and the Kueue admission path retries like a failed admission request instead of letting `catch (Exception)` turn it into the terminal `SchedulingFailure`. ### Why are the changes needed? A suspended resource whose driver or master was requested before has to complete its attempt rather than be held, so both steps read the live resource to tell the two cases apart. That read reported every failure as "never requested", so an unavailable API server was enough to release the Kueue quota reservation of a resource whose driver or master is actually running, which is what the check above that branch guards against. [SPARK-59667](https://issues.apache.org/jira/browse/SPARK-59667) made the initial read of `getOrCreateSecondaryResource` strict for the same reason, and deliberately kept `ReconcilerUtils.getResource` lenient, so these two callers ask for the strict read themselves. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
