[jira] [Comment Edited] (YARN-5718) TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior
[ https://issues.apache.org/jira/browse/YARN-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573792#comment-15573792 ] Xuan Gong edited comment on YARN-5718 at 10/14/16 1:38 AM: --- +1 LGTM. [~djp] But I am worried about the back-incompatibility issue. Could you file a separate ticket to have a discussion on branch-2 and branch-2.8 for this ticket, please ? In the mean time, I will commit the patch into trunk. was (Author: xgong): +1 LGTM. [~djp] But I am worried about the back-incompatibility issue. Could you file a separate ticket to have a discussion on branch-2 and branch-2.8 for this ticket, please ? At the mean time, I will commit the patch into trunk. > TimelineClient (and other places in YARN) shouldn't over-write HDFS client > retry settings which could cause unexpected behavior > --- > > Key: YARN-5718 > URL: https://issues.apache.org/jira/browse/YARN-5718 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineclient >Reporter: Junping Du >Assignee: Junping Du > Attachments: YARN-5718-v2.1.patch, YARN-5718-v2.patch, YARN-5718.patch > > > In one HA cluster, after NN failed over, we noticed that job is getting > failed as TimelineClient failed to retry connection to proper NN. This is > because we are overwrite hdfs client settings that hard code retry policy to > be enabled that conflict NN failed-over case - hdfs client should fail fast > so can retry on another NN. > We shouldn't assume any retry policy for hdfs client at all places in YARN. > This should keep consistent with HDFS settings that has different retry > polices in different deployment case. Thus, we should clean up these hard > code settings in YARN, include: FileSystemTimelineWriter, > FileSystemRMStateStore and FileSystemNodeLabelsStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5718) TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior
[ https://issues.apache.org/jira/browse/YARN-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573792#comment-15573792 ] Xuan Gong edited comment on YARN-5718 at 10/14/16 1:39 AM: --- +1 LGTM. [~djp] But I am worried about the back-incompatibility issue. Could you file a separate ticket to have a discussion for branch-2 and branch-2.8 ,please ? In the mean time, I will commit the patch into trunk. was (Author: xgong): +1 LGTM. [~djp] But I am worried about the back-incompatibility issue. Could you file a separate ticket to have a discussion on branch-2 and branch-2.8 for this ticket, please ? In the mean time, I will commit the patch into trunk. > TimelineClient (and other places in YARN) shouldn't over-write HDFS client > retry settings which could cause unexpected behavior > --- > > Key: YARN-5718 > URL: https://issues.apache.org/jira/browse/YARN-5718 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, timelineclient >Reporter: Junping Du >Assignee: Junping Du > Attachments: YARN-5718-v2.1.patch, YARN-5718-v2.patch, YARN-5718.patch > > > In one HA cluster, after NN failed over, we noticed that job is getting > failed as TimelineClient failed to retry connection to proper NN. This is > because we are overwrite hdfs client settings that hard code retry policy to > be enabled that conflict NN failed-over case - hdfs client should fail fast > so can retry on another NN. > We shouldn't assume any retry policy for hdfs client at all places in YARN. > This should keep consistent with HDFS settings that has different retry > polices in different deployment case. Thus, we should clean up these hard > code settings in YARN, include: FileSystemTimelineWriter, > FileSystemRMStateStore and FileSystemNodeLabelsStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
