bmarcott commented on a change in pull request #26696: [WIP][SPARK-18886][CORE]
Make locality wait time be the time since a TSM's available slots were fully
utilized
URL: https://github.com/apache/spark/pull/26696#discussion_r355951463
##########
File path:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
##########
@@ -808,6 +815,7 @@ private[spark] class TaskSchedulerImpl(
// happen below in the rootPool.executorLost() call.
taskIds.foreach(cleanupTaskState)
}
+ executorIdToCores.remove(executorId).foreach(totalSlots -= _ /
CPUS_PER_TASK)
Review comment:
updateAvailableSlots is dependent on both the total number of slots + what
tasksets/pools are present. Will take a look to see how feasible doing the
update incrementally is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
