[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323805#comment-14323805 ] Hudson commented on YARN-2994: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm * hadoop-yarn-project/CHANGES.txt Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319855#comment-14319855 ] Hudson commented on YARN-2994: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #103 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/103/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm * hadoop-yarn-project/CHANGES.txt Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319889#comment-14319889 ] Hudson commented on YARN-2994: -- FAILURE: Integrated in Hadoop-Yarn-trunk #837 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/837/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14320227#comment-14320227 ] Hudson commented on YARN-2994: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #104 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/104/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm * hadoop-yarn-project/CHANGES.txt Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14320132#comment-14320132 ] Hudson commented on YARN-2994: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2035 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2035/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm * hadoop-yarn-project/CHANGES.txt Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14320272#comment-14320272 ] Hudson commented on YARN-2994: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2054 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2054/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm * hadoop-yarn-project/CHANGES.txt Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318842#comment-14318842 ] Jian He commented on YARN-2994: --- Since work-preserving recovery is enabled by default and recommended for users, I removed the max-attempt config which is not needed for configuring work-preserving recovery. Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318837#comment-14318837 ] Jian He commented on YARN-2994: --- thanks [~djp], [~ozawa] ! I incorporated your comments. Also added a section for level-db based state store. Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[
https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318946#comment-14318946
]
Hadoop QA commented on YARN-2994:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12698513/YARN-2994.3.patch
against trunk revision 58cb9f5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6621//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6621//console
This message is automatically generated.
Document work-preserving RM restart
---
Key: YARN-2994
URL: https://issues.apache.org/jira/browse/YARN-2994
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[
https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319296#comment-14319296
]
Hadoop QA commented on YARN-2994:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12698549/YARN-2994.4.patch
against trunk revision 04d09ec.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6626//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6626//console
This message is automatically generated.
Document work-preserving RM restart
---
Key: YARN-2994
URL: https://issues.apache.org/jira/browse/YARN-2994
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch,
YARN-2994.4.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319539#comment-14319539 ] Tsuyoshi OZAWA commented on YARN-2994: -- +1, committing this shortly. Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319559#comment-14319559 ] Hudson commented on YARN-2994: -- FAILURE: Integrated in Hadoop-trunk-Commit #7101 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7101/]) YARN-2994. Document work-preserving RM restart. Contributed by Jian He. (ozawa: rev b0d81e05abd730a923aed4727a8191650aebd2f9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRestart.apt.vm * hadoop-yarn-project/CHANGES.txt Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Fix For: 2.7.0 Attachments: YARN-2994.1.patch, YARN-2994.2.patch, YARN-2994.3.patch, YARN-2994.4.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[
https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318540#comment-14318540
]
Junping Du commented on YARN-2994:
--
Thanks [~jianhe] for delivering a documentation patch for this important
feature!
Patch looks good in overall. However, some typos listed below need to be fixed.
{code}
+ Focus on re-constructing the running state of ResourceManger by combining
...
+ As of Hadoop 2.4.0 release, the behaviors for NodeMangers and
ApplicationMasters to handle this command
...
the container statuses from NodeMangers and container requests from
ApplicationMasters
...
NodeMangers and clients during the down-time of RM will keep polling RM until
RM comes up. When RM becomes alive, it will send a re-sync command to all the
NodeMangers and ApplicationMasters it was talking to via heartbeats.
...
+ As of Hadoop 2.4.0 release, the behaviors for NodeMangers and
ApplicationMasters to handle this command
{code}
ResourceManger = ResourceManager, NodeMangers = NodeManagers
Document work-preserving RM restart
---
Key: YARN-2994
URL: https://issues.apache.org/jira/browse/YARN-2994
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2994.1.patch, YARN-2994.2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[
https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318601#comment-14318601
]
Tsuyoshi OZAWA commented on YARN-2994:
--
Thanks Jian for writhing this document.
{code}
It is also can be used with FileSysytemStore also right?
{code}
This is correct, but it's a bit confusing for users to understand when to use
FileSystemRMStateStore and when to use ZKRMStateStore. It's better to write
that FileSystemRMStateStore can be used only for single node(correctly, single
process) RM setup explicitly since there are no fence mechanism in
FileSystemRMStateStore. What do you think?
Document work-preserving RM restart
---
Key: YARN-2994
URL: https://issues.apache.org/jira/browse/YARN-2994
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2994.1.patch, YARN-2994.2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[
https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317570#comment-14317570
]
Hadoop QA commented on YARN-2994:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12698263/YARN-2994.2.patch
against trunk revision 67efab9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6607//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6607//console
This message is automatically generated.
Document work-preserving RM restart
---
Key: YARN-2994
URL: https://issues.apache.org/jira/browse/YARN-2994
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2994.1.patch, YARN-2994.2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259853#comment-14259853 ] Rohith commented on YARN-2994: -- Thanks [~jianhe] for woking on documenting work preserving restart feature. I quickly read patch, changes are fine. I have one basic doubt that does work preserving restart work only for ZKRMStateStore? It is also can be used with FileSysytemStore also right? Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Attachments: YARN-2994.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259864#comment-14259864 ] Jian He commented on YARN-2994: --- yes, that's correct. Document work-preserving RM restart --- Key: YARN-2994 URL: https://issues.apache.org/jira/browse/YARN-2994 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Jian He Assignee: Jian He Attachments: YARN-2994.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2994) Document work-preserving RM restart
[
https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259213#comment-14259213
]
Hadoop QA commented on YARN-2994:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12689195/YARN-2994.1.patch
against trunk revision 40ee4bf.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6194//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6194//console
This message is automatically generated.
Document work-preserving RM restart
---
Key: YARN-2994
URL: https://issues.apache.org/jira/browse/YARN-2994
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2994.1.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
