[ https://issues.apache.org/jira/browse/MAPREDUCE-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522514#comment-15522514 ]
Jian He edited comment on MAPREDUCE-6726 at 9/26/16 9:10 AM: ------------------------------------------------------------- [~srikanth.sampath], thanks for the patch , I looked at it. IIUC, we are also going to have a different mechanism to retrieve the AM address via YARN-4758. The patch right now is hardcoded to depend on registry approach only, this part of the code needs to be made pluggable so that the approach listed in YARN-4758 can be plugged in. We could implement different FailoverProvider like RegistryBasedFailoverProvider for this jira or RPCBasedFailoverProvider for YARN-4758. Regarding the JVMId changes, could you separate that out and upload it on to MAPREDUCE-6754 ? we can get that reviewed and committed first. was (Author: jianhe): [~srikanth.sampath], thanks for the patch , I looked at it. IIUC, we are also going to have a different mechanism to retrieve the AM address via YARN-4758. The patch right now is hardcoded to depend on registry approach only, this part of the code needs to be made pluggable so that the approach listed in YARN-4758 can be plugged in. We could implement different FailoverProvider like RegistryBasedFailoverProvider or RPCBasedFailoverProvider. Regarding the JVMId changes, could you separate that out and upload it on to MAPREDUCE-6754 ? we can get that reviewed and committed first. > YARN Registry based AM discovery with retry and in-flight task persistent via > JHS > --------------------------------------------------------------------------------- > > Key: MAPREDUCE-6726 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6726 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster > Reporter: Junping Du > Assignee: Srikanth Sampath > Attachments: MAPREDUCE-6726-MAPREDUCE-6608.001.patch, > MAPREDUCE-6726-MAPREDUCE-6608.001.patch, > MAPREDUCE-6726-MAPREDUCE-6608.002.patch, WorkPreservingMRAppMaster.pdf > > > Several tasks will be achieved in this JIRA based on the demo patch in > MAPREDUCE-6608: > 1. AM discovery base on YARN register service. Could be replaced by YARN-4758 > later due to scale up issue. > 2. Retry logic for TaskUmbilicalProtocol RPC connection > 3. In-flight task recover after AM restart via JHS > 4. Configuration to control the behavior compatible with previous when not > enable this feature (by default). > All security related issues and other concerns discussed in MAPREDUCE-6608 > will be addressed in follow up JIRAs. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org