[
https://issues.apache.org/jira/browse/YARN-6511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035370#comment-16035370
]
Subru Krishnan commented on YARN-6511:
--------------------------------------
Thanks [~botong] for the patch. I looked at it & please find my comments below:
* I don't see the need for {{TestableAMRMProxyPolicy}}, we should simply
reuse {{BroadcastAMRMProxyPolicy}}. If possible, do the same for
{{TestableRouterPolicy}}.
* Do we need a {{UnmanagedAMPoolManager}} per interceptor instance or can we
use one at {{AMRMProxyService}} level?
* *loadAMRMPolicy* is independent of {{FederationInterceptor}}, can be
refactored out to common class.
* Is updating the _queue_ below safe in *loadAMRMPolicy*?
{code} queue = YarnConfiguration.DEFAULT_FEDERATION_POLICY_KEY;{code}
* I feel the *finishApplicationMaster* of the pool should be moved to
{{UnmanagedAMPoolManager}}.
* I see dynamic instantiations of {{ExecutorCompletionService}} in finish,
register, etc invocations. Wouldn't we be better served by pre-initializing it?
* Is *getSubClusterForNode* required as the resolver should be doing this
instead of every client?
* Consider looping on _registrations_ in lieu of _requests_ in
*sendRequestsToSecondaryResourceManagers* as that'll not only minimize the
iterations but also make the checks redundant.
* Quite a few ops like creation of _YarnConfiguration_,
_getApplicationContext_ invocation etc can be moved outside the for loop in
*registerWithNewSubClusters*.
* {{YarnConfiguration.getClusterId(getConf())}} is unnecessary as we have
_homeSubClusterId_.
> Federation Intercepting and propagating AM-RM communications (part two:
> secondary subclusters added)
> ----------------------------------------------------------------------------------------------------
>
> Key: YARN-6511
> URL: https://issues.apache.org/jira/browse/YARN-6511
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Botong Huang
> Assignee: Botong Huang
> Attachments: YARN-6511-YARN-2915.v1.patch,
> YARN-6511-YARN-2915.v2.patch
>
>
> In order to support transparent "spanning" of jobs across sub-clusters, all
> AM-RM communications are proxied (via YARN-2884).
> This JIRA tracks federation-specific mechanisms that decide how to
> "split/broadcast" requests to the RMs and "merge" answers to
> the AM.
> This the part two jira, which adds secondary subclusters and do full
> split-merge for requests. Part one is in YARN-3666
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
