Anubhav Dhoot commented on YARN-3676:

Thanks [~asuresh] for working on this. I see the patch continues assigning on 
the node if you have *any* app which has a specific request on that node. But 
the scheduling attempt (via queueMgr.getRootQueue().assignContainer(node)) does 
not restrict which apps will get allocation on that node. So one could end up 
assigning the next container on the node for an app which may not have a 
specific request for that node. 
I see two choices. 
a) Smaller change - We should allow subsequent assignments only for the node 
local only apps? And you already have that list in the map. That can end up 
prioritizing the application's node local request over other applications.
b) BIgger change - Once we have picked the app based on priority, we allow it 
to assign multiple containers if there are multiple node local requests for 
that node.

> Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL 
> resource requests
> --------------------------------------------------------------------------------------------
>                 Key: YARN-3676
>                 URL: https://issues.apache.org/jira/browse/YARN-3676
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-3676.1.patch, YARN-3676.2.patch, YARN-3676.3.patch, 
> YARN-3676.4.patch, YARN-3676.5.patch
> AssignMultiple is generally set to false to prevent overloading a Node (for 
> eg, new NMs that have just joined)
> A possible scheduling optimization would be to disregard this directive for 
> apps whose allowed locality is NODE_LOCAL

This message was sent by Atlassian JIRA

Reply via email to