bmarcott commented on a change in pull request #27207:
URL: https://github.com/apache/spark/pull/27207#discussion_r413542822
##########
File path:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
##########
@@ -319,20 +336,38 @@ private[spark] class TaskSchedulerImpl(
taskSetsByStageIdAndAttempt -= manager.taskSet.stageId
}
}
+ resetOnPreviousOffer -= manager.taskSet
manager.parent.removeSchedulable(manager)
logInfo(s"Removed TaskSet ${manager.taskSet.id}, whose tasks have all
completed, from pool" +
s" ${manager.parent.name}")
}
+ /**
+ * Offers resources to a single [[TaskSetManager]] at a given max allowed
[[TaskLocality]].
+ *
+ * @param taskSet task set manager to offer resources to
+ * @param maxLocality max locality to allow when scheduling
+ * @param shuffledOffers shuffled resource offers to use for scheduling,
+ * remaining resources are tracked by below fields as
tasks are scheduled
+ * @param availableCpus remaining cpus per offer,
+ * value at index 'i' corresponds to shuffledOffers[i]
+ * @param availableResources remaining resources per offer,
+ * value at index 'i' corresponds to
shuffledOffers[i]
+ * @param tasks tasks scheduled per offer, value at index 'i' corresponds to
shuffledOffers[i]
+ * @param addressesWithDescs tasks scheduler per host:port, used for barrier
tasks
+ * @return tuple of (had delay schedule rejects?, option of min locality of
launched task)
Review comment:
thanks for catching. The code changed mid PR and missed updating the doc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
