Karthik Kambatla commented on YARN-3676:

Thanks for working on this, [~asuresh]. Comments:
# What do you think of a new class to maintain details about node-local-only 
requests to avoid complicating the already-complex FairScheduler class? This 
would essentially host the concurrent-hash-map here, and methods to 
{{addIfNodeLocalOnlyRequest}}, {{removeIfNodeLocalOnlyRequest}} and 
# Also, would be nice to comment on what the map is actually mapping, since the 
key for outer-map is a String.
# While examining each request to see if it is node local only, 
## it appears we include rack-local-only requests as well. Can we add a test to 
ensure this doesn't happen? 
## we seem to be creating a ConcurrentHashMap for the app always. We should 
create this only if one hasn't already been created.

> Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL 
> resource requests
> --------------------------------------------------------------------------------------------
>                 Key: YARN-3676
>                 URL: https://issues.apache.org/jira/browse/YARN-3676
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-3676.1.patch, YARN-3676.2.patch, YARN-3676.3.patch, 
> YARN-3676.4.patch, YARN-3676.5.patch
> AssignMultiple is generally set to false to prevent overloading a Node (for 
> eg, new NMs that have just joined)
> A possible scheduling optimization would be to disregard this directive for 
> apps whose allowed locality is NODE_LOCAL

This message was sent by Atlassian JIRA

Reply via email to