attilapiros commented on a change in pull request #33492:
URL: https://github.com/apache/spark/pull/33492#discussion_r675409750
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
##########
@@ -264,21 +272,22 @@ private[spark] class ExecutorPodsAllocator(
//
// TODO: with dynamic allocation off, handle edge cases if we end up
with more running
// executors than expected.
- val knownPodCount = currentRunningCount +
+ var notRunningPodCountForRpId =
currentPendingExecutorsForRpId.size +
schedulerKnownPendingExecsForRpId.size +
newlyCreatedExecutorsForRpId.size +
schedulerKnownNewlyCreatedExecsForRpId.size
+ val podCountForRpId = currentRunningCount + notRunningPodCountForRpId
- if (knownPodCount > targetNum) {
- val excess = knownPodCount - targetNum
+ if (podCountForRpId > targetNum) {
+ val excess = podCountForRpId - targetNum
val newlyCreatedToDelete = newlyCreatedExecutorsForRpId
.filter { case (_, (_, createTime)) =>
currentTime - createTime > executorIdleTimeout
}.keys.take(excess).toList
- val knownPendingToDelete = currentPendingExecutorsForRpId
+ val pendingToDelete = currentPendingExecutorsForRpId
Review comment:
Last rename for the same reason as earlier: this are PODs unknow by the
scheduler so safe to be removed here (no task can be scheduled on them).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
