Ngone51 commented on code in PR #36716:
URL: https://github.com/apache/spark/pull/36716#discussion_r890982027
##########
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##########
@@ -725,26 +729,38 @@ private[deploy] class Master(
*/
private def startExecutorsOnWorkers(): Unit = {
// Right now this is a very simple FIFO scheduler. We keep trying to fit
in the first app
- // in the queue, then the second app, etc.
+ // in the queue, then the second app, etc. And for each app, we will
schedule base on
+ // resource profiles also with a simple FIFO scheduler, resource profile
with smaller id
+ // first.
Review Comment:
I'd suggest to schedule in the order of the resource profile reuqests
instead of the smaller id first. In the case of the resource profile is resued
for later on RDD computation, the samller id doesn't seem to has the priority
over other resource profiles. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
