[
https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403189#comment-15403189
]
Ellen Hui edited comment on YARN-5390 at 8/2/16 10:38 PM:
----------------------------------------------------------
Hi [~leftnoteasy], thanks for the quick feedback!
* This interface will be used in the three patches you looked at, although you
are correct that they have not been updated yet. For instance, the
LocalityMulticastAMRMProxyFederationPolicy prototype in YARN-5325 uses the
FederationSubClusterResolver interface to split resource requests. There are
some examples of the resolver being used in the splitResourceRequests method of
that class, although some of the classnames are out of date. From the javadoc:
{panel}
host localized ResourceRequest are always forwarded to the RM
that owns the node, based on the feedback of a FederationSubClusterResolver
rack localized ResourceRequest are forwarded to the RM that owns
the rack (if the FederationSubClusterResolver provides this info) or
they are forwarded as if they were ANY (this is important for deployment that
stripe racks across sub-clusters) as there is not a single resolution.
ANY request corresponding to node/rack local requests are only forwarded
to the set of RMs that owns the node-local requests. The number of containers
listed in each ANY is proportional to the number of node-local container
requests (associated to this ANY via the same allocateRequestId)
{panel}
* The FederationInterceptor from YARN-5325 will be responsible for managing the
lifecyle of the SubClusterResolver.
* I think it's better to leave the SubClusterResolver methods non-static, since
we want to allow the implementation to be pluggable and I can't think of a
particular reason it should be static. Please let me know if you disagree, I
may be missing something.
Thanks!
was (Author: ellenfkh):
Hi [~wangda], thanks for the quick feedback!
This interface will be used in the three patches you looked at, although you
are correct that they have not been updated yet. For instance, the
LocalityMulticastAMRMProxyFederationPolicy prototype in YARN-5325 uses the
FederationSubClusterResolver interface to split resource requests. From the
javadoc:
host localized ResourceRequest are always forwarded to the RM
that owns the node, based on the feedback of a FederationSubClusterResolver
rack localized ResourceRequest are forwarded to the RM that owns
the rack (if the FederationSubClusterResolver provides this info) or
they are forwarded as if they were ANY (this is important for deployment that
stripe racks across sub-clusters) as there is not a single resolution.
ANY request corresponding to node/rack local requests are only forwarded
to the set of RMs that owns the node-local requests. The number of containers
listed in each ANY is proportional to the number of node-local container
requests (associated to this ANY via the same allocateRequestId)
There are some examples of the resolver being used in the splitResourceRequests
method of the same class, although some of the classnames are out of date.
The FederationInterceptor from YARN-5325 will be responsible for managing the
lifecyle of the SubClusterResolver.
I think it's better to leave the SubClusterResolver methods non-static, since
we want to allow the implementation to be pluggable and I can't think of a
particular reason it should be static. Please let me know if you disagree, I
may be missing something.
Thanks!
> Federation Subcluster Resolver
> ------------------------------
>
> Key: YARN-5390
> URL: https://issues.apache.org/jira/browse/YARN-5390
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Carlo Curino
> Assignee: Ellen Hui
> Attachments: YARN-5390-YARN-2915.v0.patch,
> YARN-5390-YARN-2915.v1.patch, YARN-5390-YARN-2915.v2.patch
>
>
> This JIRA tracks effort to create a mechanism to resolve nodes/racks resource
> names to sub-cluster identifiers. This is needed by the federation policies
> in YARN-5323, YARN-5324, YARN-5325 to operate correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
