HwangDongJun commented on code in PR #58187:
URL: https://github.com/apache/spark/pull/58187#discussion_r4310544094
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala:
##########
@@ -209,6 +257,26 @@ private[spark] class Client(
log"submission ID ${MDC(SUBMISSION_ID, sId)} into Kubernetes")
}
}
+
+ // SPARK-38079: best-effort cleanup for the two failure catch blocks between
pod creation and
+ // gate removal. The pod and each pre-resource are deleted independently --
every delete call
+ // wrapped in its own Utils.tryLogNonFatalError -- so that one of them
failing (e.g. a delete
+ // call itself hitting a permission or network error) neither masks the
original exception
+ // the caller is about to (re)throw, nor skips the rest. In particular,
fabric8's
+ // resourceList(...).delete() deletes the given items sequentially and stops
at the first
+ // non-404 exception, so pre-resources are deleted one at a time here rather
than as a single
+ // resourceList(...).delete() call, which could otherwise leave every
pre-resource after the
+ // one that failed undeleted.
+ private def deletePodAndPreResources(pod: Pod, preResources:
Seq[HasMetadata]): Unit = {
+ Utils.tryLogNonFatalError {
+ kubernetesClient.pods().resource(pod).delete()
Review Comment:
Guarded the same way: the pod delete in the post-resource catch block is now
inside `Utils.tryLogNonFatalError`. `SPARK-38079: a failure deleting the pod
does not hide the original failure` covers it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
