[
https://issues.apache.org/jira/browse/MAPREDUCE-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522514#comment-15522514
]
Jian He edited comment on MAPREDUCE-6726 at 9/26/16 9:10 AM:
-------------------------------------------------------------
[~srikanth.sampath], thanks for the patch , I looked at it.
IIUC, we are also going to have a different mechanism to retrieve the AM
address via YARN-4758. The patch right now is hardcoded to depend on registry
approach only, this part of the code needs to be made pluggable so that the
approach listed in YARN-4758 can be plugged in. We could implement different
FailoverProvider like RegistryBasedFailoverProvider for this jira or
RPCBasedFailoverProvider for YARN-4758.
Regarding the JVMId changes, could you separate that out and upload it on to
MAPREDUCE-6754 ? we can get that reviewed and committed first.
was (Author: jianhe):
[~srikanth.sampath], thanks for the patch , I looked at it.
IIUC, we are also going to have a different mechanism to retrieve the AM
address via YARN-4758. The patch right now is hardcoded to depend on registry
approach only, this part of the code needs to be made pluggable so that the
approach listed in YARN-4758 can be plugged in. We could implement different
FailoverProvider like RegistryBasedFailoverProvider or
RPCBasedFailoverProvider.
Regarding the JVMId changes, could you separate that out and upload it on to
MAPREDUCE-6754 ? we can get that reviewed and committed first.
> YARN Registry based AM discovery with retry and in-flight task persistent via
> JHS
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6726
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6726
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster
> Reporter: Junping Du
> Assignee: Srikanth Sampath
> Attachments: MAPREDUCE-6726-MAPREDUCE-6608.001.patch,
> MAPREDUCE-6726-MAPREDUCE-6608.001.patch,
> MAPREDUCE-6726-MAPREDUCE-6608.002.patch, WorkPreservingMRAppMaster.pdf
>
>
> Several tasks will be achieved in this JIRA based on the demo patch in
> MAPREDUCE-6608:
> 1. AM discovery base on YARN register service. Could be replaced by YARN-4758
> later due to scale up issue.
> 2. Retry logic for TaskUmbilicalProtocol RPC connection
> 3. In-flight task recover after AM restart via JHS
> 4. Configuration to control the behavior compatible with previous when not
> enable this feature (by default).
> All security related issues and other concerns discussed in MAPREDUCE-6608
> will be addressed in follow up JIRAs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
