[
https://issues.apache.org/jira/browse/YARN-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323233#comment-15323233
]
Subru Krishnan edited comment on YARN-4879 at 6/9/16 8:11 PM:
--------------------------------------------------------------
There was a discussion regarding the implementation details with
[~leftnoteasy], [~vinodkv], [~kasha] and [~asuresh]. The crux of the discussion
was:
* The request ID can be used for either identifying a single RR or a group of
RRs. The latter is required if we want to encode the notion of relaxLocality -
i.e. allocate a container in node N1, if not fallback to rack R1 and finally
ANY. I will fix the RR API Javdocs in YARN-4887 accordingly.
* The existing *AMRMClientImpl* is convoluted and has some vestigial
behaviour that results in auto expansion of client RRs due to rack inference.
Quite a bit of the complexity in *AMRMClientImpl* results from the matching
logic which can be greatly simplified by using the request ID. So the consensus
was to leave the existing *AMRMClientImpl* as-is for backward compatibility
and start with a fresh version of *AMRMClientImpl (v2)* that will have a clean
implementation using the request ID. I created YARN-5225 to track the first
version of *AMRMClientImpl (v2)*
was (Author: subru):
There was a discussion regarding the implementation details with
[~leftnoteasy], [~vinodkv], [~kasha] and [~asuresh]. The crux of the discussion
was:
* The request ID can be used for either identifying a single RR or a group of
RRs. The latter is required if we want to encode the notion of relaxLocality -
i.e. allocate a container in node N1, if not fallback to rack R1 and finally
ANY. I will fix the RR API Javdocs in YARN-4887 accordingly.
* The existing *AMRMClientImpl* is convoluted and has some vestigial
behaviour that results in auto expansion of client RRs due to rack inference.
Quite a bit of the complexity in *AMRMClientImpl* results from the matching
logic which can be greatly simplified by using the request ID. So the consensus
was to leave the existing *AMRMClientImpl* as-is for backward compatibility
and start with a fresh version of *AMRMClientImpl (v2)* that will have a clean
implementation using the request ID.
> Enhance Allocate Protocol to Identify Requests Explicitly
> ---------------------------------------------------------
>
> Key: YARN-4879
> URL: https://issues.apache.org/jira/browse/YARN-4879
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: applications, resourcemanager
> Reporter: Subru Krishnan
> Assignee: Subru Krishnan
> Attachments: SimpleAllocateProtocolProposal-v1.pdf,
> SimpleAllocateProtocolProposal-v2.pdf
>
>
> For legacy reasons, the current allocate protocol expects expanded requests
> which represent the cumulative request for any change in resource
> constraints. This is not only very difficult to comprehend but makes it
> impossible for the scheduler to associate container allocations to the
> original requests. This problem is amplified by the fact that the expansion
> is managed by the AMRMClient which makes it cumbersome for non-Java clients
> as they all have to replicate the non-trivial logic. In this JIRA, we are
> proposing enhancement to the Allocate Protocol to allow AMs to identify
> requests explicitly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
