[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936141#comment-13936141 ] Hudson commented on MAPREDUCE-5769: --- FAILURE: Integrated in Hadoop-Yarn-trunk #510 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/510/]) MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1577647) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMCommunicator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Fix For: 2.4.0 Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936190#comment-13936190 ] Hudson commented on MAPREDUCE-5769: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1702 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1702/]) MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1577647) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMCommunicator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Fix For: 2.4.0 Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936213#comment-13936213 ] Hudson commented on MAPREDUCE-5769: --- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1727 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1727/]) MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1577647) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMCommunicator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Fix For: 2.4.0 Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13935349#comment-13935349 ] Jason Lowe commented on MAPREDUCE-5769: --- Sorry for the delay, finally getting back to this. +1 lgtm. Committing this. Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13935402#comment-13935402 ] Hudson commented on MAPREDUCE-5769: --- SUCCESS: Integrated in Hadoop-trunk-Commit #5330 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5330/]) MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1577647) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMCommunicator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Fix For: 2.4.0 Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13919007#comment-13919007 ] Rohith commented on MAPREDUCE-5769: --- Please find more information on patch. bq. TestJobEndNotifier is not an appropriate place for these tests as they have nothing to do with job end notification I added test case in TestRMContainerAllocator. Is it okay? bq. The test should setup an appropriately mocked scheduler member and call the real register() and unregister() methods to verify that the mocked scheduler sees unregistration when it should and not when it shouldn't Fixed, test case check for isApplicationMasterRegistered. If we remove this flag from register(), test case will fail. bq. isApplicationMasterRegistered should not be exposed via get/set methods and probably doesn't need to be volatile. Fixed. For testing purpose, added protected method isApplicationMasterRegistered(). Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[
https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13915864#comment-13915864
]
Hadoop QA commented on MAPREDUCE-5769:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12631722/MAPREDUCE-5769.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4385//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4385//console
This message is automatically generated.
Unregistration to RM should not be called if AM is crashed before registering
with RM
-
Key: MAPREDUCE-5769
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Rohith
Assignee: Rohith
Attachments: MAPREDUCE-5769.1.patch, MAPREDUCE-5769.2.patch
Got the scenario from YARN-1752, where unregistration of application master
is called before registering with RM.
There should be mechanism to identify isApplicationMasaterRegistered before
calling unregistration (finishApplicationMaster).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[
https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914229#comment-13914229
]
Hadoop QA commented on MAPREDUCE-5769:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12631478/MAPREDUCE-5769.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4383//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4383//console
This message is automatically generated.
Unregistration to RM should not be called if AM is crashed before registering
with RM
-
Key: MAPREDUCE-5769
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Rohith
Assignee: Rohith
Attachments: MAPREDUCE-5769.1.patch
Got the scenario from YARN-1752, where unregistration of application master
is called before registering with RM.
There should be mechanism to identify isApplicationMasaterRegistered before
calling unregistration (finishApplicationMaster).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5769) Unregistration to RM should not be called if AM is crashed before registering with RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914702#comment-13914702 ] Jason Lowe commented on MAPREDUCE-5769: --- Thanks for reporting and tracking this down, Rohith! Comments on the patch: - TestJobEndNotifier is not an appropriate place for these tests as they have nothing to do with job end notification. - The unit test isn't really testing the fix but the fix as applied to the mocked object. A key indication is that we can remove the setApplicationMasterRegistered() call in RMCommunicator and the test will still pass. The test should setup an appropriately mocked scheduler member and call the real register() and unregister() methods to verify that the mocked scheduler sees unregistration when it should and not when it shouldn't. - isApplicationMasterRegistered should not be exposed via get/set methods and probably doesn't need to be volatile. Unregistration to RM should not be called if AM is crashed before registering with RM - Key: MAPREDUCE-5769 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5769 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rohith Assignee: Rohith Attachments: MAPREDUCE-5769.1.patch Got the scenario from YARN-1752, where unregistration of application master is called before registering with RM. There should be mechanism to identify isApplicationMasaterRegistered before calling unregistration (finishApplicationMaster). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
