[
https://issues.apache.org/jira/browse/YARN-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243067#comment-16243067
]
Botong Huang commented on YARN-6128:
------------------------------------
V6 patch uploaded addressing [~subru]'s comments.
bq. My question is why can't we get the tokens for the sub-clusters also in a
single call, to avoid the read in a loop?
The registry api doesn't allow fetching multiple keys in one call, we have to
read one at a time.
bq. I don't see the Credentials used anywhere in FSRegistryOperationsService or
available in the parent RegistryOperations interface. So maybe we can add it
when we require it? I am also concerned about its expensive retrieval
The credentials is loaded into the ugi used by {{FederationInterptor}} in line
192 in the v5 patch. If the registry is store based, it can access the tokens
from there. For recovery, we are not storing an extra copy of the credential in
NMSS, the recovered AM container already have it. Please see AMRMProxy line
243-258 in v5 patch.
bq. Thanks for the clarification but shouldn't we do it only if AM supports
recovery and if it's not the first attempt?
True, so in {{FederationInterptor.reAttachUAMAndMergeRegisterResponse}} when
recovery is not supported, registryClient will be null. If it is the first
attempt, registry won't have any entries for this app. I've updated the patch
is skip creating the threadpool for this case, as well as more comments to
explain this scenario.
> Add support for AMRMProxy HA
> ----------------------------
>
> Key: YARN-6128
> URL: https://issues.apache.org/jira/browse/YARN-6128
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: amrmproxy, nodemanager
> Reporter: Subru Krishnan
> Assignee: Botong Huang
> Attachments: YARN-6128.v0.patch, YARN-6128.v1.patch,
> YARN-6128.v1.patch, YARN-6128.v2.patch, YARN-6128.v3.patch,
> YARN-6128.v3.patch, YARN-6128.v4.patch, YARN-6128.v5.patch
>
>
> YARN-556 added the ability for RM failover without loosing any running
> applications. In a Federated YARN environment, there's additional state in
> the {{AMRMProxy}} to allow for spanning across multiple sub-clusters, so we
> need to enhance {{AMRMProxy}} to support HA.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
