[
https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923316#comment-15923316
]
Wangda Tan commented on YARN-6050:
----------------------------------
[~rkanter],
bq. If you think it would be better to use not change the label manager code
and instead filter out the :0 nodes, I can also do that instead.
I'm OK with changes of RMNodeLabelsManager. But should we use concurrent set in
RMNodeLabel? Which can avoid copy of the whole set and it will be more
consistent to latest set? (Copy and read the Set cannot capture later changes
of label-node mappings, but inconsistency result only happens to concurrent Set
when iterating and writing happen at the same time).
Some other comments:
1) RMAppManager#validateAndCreateResourceRequest, could you move ANY request to
first of the list? Since there're some assumptions in other logics like:
- {{RMAppImpl#getAmNodeLabelExpression}}:
{{getAMResourceRequests().get(0).getNodeLabelExpression()}}
- {{RMServerUtils#getApplicableNodeCountForAM}}:
{{amReqs.get(0).getNodeLabelExpression()}}
Rest of the patch looks good to me.
Considering size of the patch, I would prefer to have another set of eyes to
look at it as well. [~kasha]/[~asuresh].
> AMs can't be scheduled on racks or nodes
> ----------------------------------------
>
> Key: YARN-6050
> URL: https://issues.apache.org/jira/browse/YARN-6050
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.9.0, 3.0.0-alpha2
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: YARN-6050.001.patch, YARN-6050.002.patch,
> YARN-6050.003.patch, YARN-6050.004.patch, YARN-6050.005.patch,
> YARN-6050.006.patch, YARN-6050.007.patch, YARN-6050.008.patch,
> YARN-6050.009.patch, YARN-6050.010.patch
>
>
> Yarn itself supports rack/node aware scheduling for AMs; however, there
> currently are two problems:
> # To specify hard or soft rack/node requests, you have to specify more than
> one {{ResourceRequest}}. For example, if you want to schedule an AM only on
> "rackA", you have to create two {{ResourceRequest}}, like this:
> {code}
> ResourceRequest.newInstance(PRIORITY, ANY, CAPABILITY, NUM_CONTAINERS, false);
> ResourceRequest.newInstance(PRIORITY, "rackA", CAPABILITY, NUM_CONTAINERS,
> true);
> {code}
> The problem is that the Yarn API doesn't actually allow you to specify more
> than one {{ResourceRequest}} in the {{ApplicationSubmissionContext}}. The
> current behavior is to either build one from {{getResource}} or directly from
> {{getAMContainerResourceRequest}}, depending on if
> {{getAMContainerResourceRequest}} is null or not. We'll need to add a third
> method, say {{getAMContainerResourceRequests}}, which takes a list of
> {{ResourceRequest}} so that clients can specify the multiple resource
> requests.
> # There are some places where things are hardcoded to overwrite what the
> client specifies. These are pretty straightforward to fix.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
