Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23136#discussion_r237626777
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
---
@@ -86,11 +88,14 @@ private[spark] class ExecutorPodsAllocator(
s" cluster after $podCreationTimeout milliseconds despite the
fact that a" +
" previous allocation attempt tried to create it. The executor
may have been" +
" deleted but the application missed the deletion event.")
- Utils.tryLogNonFatalError {
- kubernetesClient
- .pods()
- .withLabel(SPARK_EXECUTOR_ID_LABEL, execId.toString)
- .delete()
+
+ if (shouldDeleteExecutors) {
+ Utils.tryLogNonFatalError {
+ kubernetesClient
+ .pods()
+ .withLabel(SPARK_EXECUTOR_ID_LABEL, execId.toString)
--- End diff --
Agreed this bug fix should be in a separate PR and should be backport to
2.4. @skonto can you create a separate PR with a Jira for the bug fix? Thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
