Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/23136#discussion_r236089683
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -82,11 +85,13 @@ private[spark] class KubernetesClusterSchedulerBackend(
pollEvents.stop()
}
- Utils.tryLogNonFatalError {
- kubernetesClient.pods()
- .withLabel(SPARK_APP_ID_LABEL, applicationId())
- .withLabel(SPARK_ROLE_LABEL, SPARK_POD_EXECUTOR_ROLE)
- .delete()
+ if (shouldDeleteExecutors) {
+ Utils.tryLogNonFatalError {
+ kubernetesClient.pods()
+ .withLabel(SPARK_APP_ID_LABEL, applicationId())
+ .withLabel(SPARK_ROLE_LABEL, SPARK_POD_EXECUTOR_ROLE)
+ .delete()
+ }
--- End diff --
safest scenario as the backend stops all executors before pod deletion.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
