[
https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796422#comment-15796422
]
Wangda Tan commented on YARN-6050:
----------------------------------
[~rkanter],
Make sense to me. Reconsidered about this, overload semantics of
ResourceRequest is not a good idea.
For 001 patch we have two fields for AM request inside ASC, and it added a new
field to proto as well. I think we can improve this further:
1. It's better to deprecate the original get/set single ResourceRequest API,
which to force applications move to new API. And we can also avoid our
scheduler internal implementation to use the field.
2. We don't have to add new field to proto, according to PB compatibility doc:
https://developers.google.com/protocol-buffers/docs/proto#backwards-compatibility,
changing from optional to repeated is compatible:
bq. optional is compatible with repeated. Given serialized data of a repeated
field as input, clients that expect this field to be optional will take the
last input value if it's a primitive type field or merge all input elements if
it's a message type field.
We can add small checks to ResourceRequestPBImpl to handle this. Rest of the
logics generally looks good to me, and it's better to add a CS test as well.
> AMs can't be scheduled on racks or nodes
> ----------------------------------------
>
> Key: YARN-6050
> URL: https://issues.apache.org/jira/browse/YARN-6050
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.9.0, 3.0.0-alpha2
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: YARN-6050.001.patch
>
>
> Yarn itself supports rack/node aware scheduling for AMs; however, there
> currently are two problems:
> # To specify hard or soft rack/node requests, you have to specify more than
> one {{ResourceRequest}}. For example, if you want to schedule an AM only on
> "rackA", you have to create two {{ResourceRequest}}, like this:
> {code}
> ResourceRequest.newInstance(PRIORITY, ANY, CAPABILITY, NUM_CONTAINERS, false);
> ResourceRequest.newInstance(PRIORITY, "rackA", CAPABILITY, NUM_CONTAINERS,
> true);
> {code}
> The problem is that the Yarn API doesn't actually allow you to specify more
> than one {{ResourceRequest}} in the {{ApplicationSubmissionContext}}. The
> current behavior is to either build one from {{getResource}} or directly from
> {{getAMContainerResourceRequest}}, depending on if
> {{getAMContainerResourceRequest}} is null or not. We'll need to add a third
> method, say {{getAMContainerResourceRequests}}, which takes a list of
> {{ResourceRequest}} so that clients can specify the multiple resource
> requests.
> # There are some places where things are hardcoded to overwrite what the
> client specifies. These are pretty straightforward to fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
