attilapiros commented on a change in pull request #31513:
URL: https://github.com/apache/spark/pull/31513#discussion_r582672797
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
##########
@@ -164,15 +180,16 @@ private[spark] class ExecutorPodsAllocator(
_deletedExecutorIds = _deletedExecutorIds.filter(existingExecs.contains)
}
+ val notDeletedPods =
lastSnapshot.executorPods.filterKeys(!_deletedExecutorIds.contains(_))
Review comment:
This fixes an extra bug I found during testing. Without this change the
5.) point in my new test would fail as those pending pods which requested to be
deleted earlier by the executor PODs allocator counted as available pending
PODs for this resource profile (into the `currentPendingExecutorsForRpId`) and
upscale is not triggered at:
```
if (knownPodCount > targetNum) {
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
