[
https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803580#comment-15803580
]
Wangda Tan commented on YARN-6050:
----------------------------------
Thanks [~rkanter],
{code}
474 // Make sure that all of the requests agree with the ANY request
475 for (ResourceRequest amReq : amReqs) {
476 amReq.setCapability(anyReq.getCapability());
477
amReq.setExecutionTypeRequest(anyReq.getExecutionTypeRequest());
478 amReq.setNumContainers(anyReq.getNumContainers());
479 amReq.setPriority(anyReq.getPriority());
480 }
{code}
I think it might be better to hardcode executionType (guaranteed),
numContainers (1), priority (0). Correct?
And also, had an offline discussion with
[~vinodkv]/[~djp]/[~jianhe]/[~gtCarrera9].
A quick summary of what we discussed:
- This is more like a feature to help troubleshooting (like enforce AM launches
on a specific host to isolate the problem).
- Node label request can help placement as well.
- Need to be extra carefully about hard locality vs. AM blacklisting.
- To avoid AM container cannot be allocated because of hard locality, should we
add an option to enable/disable the feature?
[~vinodkv]/[~djp]/[~jianhe]/[~gtCarrera9] please add your comments if I missed
anything.
> AMs can't be scheduled on racks or nodes
> ----------------------------------------
>
> Key: YARN-6050
> URL: https://issues.apache.org/jira/browse/YARN-6050
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.9.0, 3.0.0-alpha2
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: YARN-6050.001.patch, YARN-6050.002.patch,
> YARN-6050.003.patch, YARN-6050.004.patch
>
>
> Yarn itself supports rack/node aware scheduling for AMs; however, there
> currently are two problems:
> # To specify hard or soft rack/node requests, you have to specify more than
> one {{ResourceRequest}}. For example, if you want to schedule an AM only on
> "rackA", you have to create two {{ResourceRequest}}, like this:
> {code}
> ResourceRequest.newInstance(PRIORITY, ANY, CAPABILITY, NUM_CONTAINERS, false);
> ResourceRequest.newInstance(PRIORITY, "rackA", CAPABILITY, NUM_CONTAINERS,
> true);
> {code}
> The problem is that the Yarn API doesn't actually allow you to specify more
> than one {{ResourceRequest}} in the {{ApplicationSubmissionContext}}. The
> current behavior is to either build one from {{getResource}} or directly from
> {{getAMContainerResourceRequest}}, depending on if
> {{getAMContainerResourceRequest}} is null or not. We'll need to add a third
> method, say {{getAMContainerResourceRequests}}, which takes a list of
> {{ResourceRequest}} so that clients can specify the multiple resource
> requests.
> # There are some places where things are hardcoded to overwrite what the
> client specifies. These are pretty straightforward to fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
