venkata91 commented on a change in pull request #28287:
URL: https://github.com/apache/spark/pull/28287#discussion_r449798608
##########
File path: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala
##########
@@ -829,12 +868,25 @@ private[spark] class ExecutorAllocationManager(
numTotalTasks - numRunning
}
+ def pendingUnschedulableTasksPerResourceProfile(rp: Int): Int = {
Review comment:
In the blacklisting logic,
https://github.com/apache/spark/blob/263f04db865920d9c10251517b00a1b477b58ff1/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala#L615
- here we just check for the first task which is currently unschedulable
(note: there could be more which we would eventually come back to), which is
why we will only have 1 unschedulable task per taskset, thats why only we keep
track of the unschedulableTaskSets and not the individual tasks in the taskSets
themselves. So in short # of unschedulableTaskSets = # of unschedulableTasks fr
now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
