[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[ https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075930#comment-14075930 ] Jian He commented on YARN-2209: --- bq. there could be possibly loosing HeartbeatThread if again responseQueue.add(response); InterruptedException. Can it be in while loop? I see. added the while loop back. bq. can add Note to AM_SHUTDOWN that providing link to ApplicationNotFoundException. add description too. Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart Key: YARN-2209 URL: https://issues.apache.org/jira/browse/YARN-2209 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Jian He Attachments: YARN-2209.1.patch, YARN-2209.2.patch, YARN-2209.3.patch, YARN-2209.4.patch, YARN-2209.5.patch YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate application to re-register on RM restart. we should do the same for AMS#allocate call also. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[ https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075877#comment-14075877 ] Rohith commented on YARN-2209: -- Thanks Jian He for updating patch. It looks good overall to me. Some comments 1. CMIIW, there could be possibly loosing HeartbeatThread if again responseQueue.add(response); InterruptedException. Can it be in while loop? please provide your thoughts. 2. In AMCommand,similar to AM_RESYNC, can add Note to AM_SHUTDOWN that providing link to ApplicationNotFoundException. Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart Key: YARN-2209 URL: https://issues.apache.org/jira/browse/YARN-2209 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Jian He Attachments: YARN-2209.1.patch, YARN-2209.2.patch, YARN-2209.3.patch, YARN-2209.4.patch YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate application to re-register on RM restart. we should do the same for AMS#allocate call also. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[
https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074366#comment-14074366
]
Rohith commented on YARN-2209:
--
Hi [~jianhe], I reviewed patch and found some comments
1. Missing lastResponseID=0 in RMContainerAllocator#getResources().
{code}
catch (ApplicationMasterNotRegisteredException e) {
LOG.info(ApplicationMaster is out of sync with ResourceManager,
+ hence resync and send outstanding requests.);
// RM may have restarted, re-register with RM.
register();
addOutstandingRequestOnResync();
return null;
}
{code}
2. In AMRMClientAsyncImpl, below code may loose one response since it is not
adding back to responseQueue when InterruptedException ocure. This may be worst
case, but still it can ocure may because java itself Interrupting or os may be
Interrupting.
Can we add reponse back to responseQueue on InterruptedException?
{code}
if (response != null) {
try {
responseQueue.put(response);
break;
} catch (InterruptedException ex) {
LOG.debug(Interrupted while waiting to put on response queue,
ex);
}
{code}
Replace allocate#resync command with ApplicationMasterNotRegisteredException
to indicate AM to re-register on RM restart
Key: YARN-2209
URL: https://issues.apache.org/jira/browse/YARN-2209
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2209.1.patch, YARN-2209.2.patch, YARN-2209.3.patch
YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate
application to re-register on RM restart. we should do the same for
AMS#allocate call also.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[ https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075125#comment-14075125 ] Jian He commented on YARN-2209: --- Thanks for the review, Rohith. Uploaded a new patch which fixed the above comments. Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart Key: YARN-2209 URL: https://issues.apache.org/jira/browse/YARN-2209 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Jian He Attachments: YARN-2209.1.patch, YARN-2209.2.patch, YARN-2209.3.patch, YARN-2209.4.patch YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate application to re-register on RM restart. we should do the same for AMS#allocate call also. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[
https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075172#comment-14075172
]
Hadoop QA commented on YARN-2209:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657938/YARN-2209.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4440//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4440//console
This message is automatically generated.
Replace allocate#resync command with ApplicationMasterNotRegisteredException
to indicate AM to re-register on RM restart
Key: YARN-2209
URL: https://issues.apache.org/jira/browse/YARN-2209
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2209.1.patch, YARN-2209.2.patch, YARN-2209.3.patch,
YARN-2209.4.patch
YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate
application to re-register on RM restart. we should do the same for
AMS#allocate call also.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[
https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073918#comment-14073918
]
Hadoop QA commented on YARN-2209:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657737/YARN-2209.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4422//console
This message is automatically generated.
Replace allocate#resync command with ApplicationMasterNotRegisteredException
to indicate AM to re-register on RM restart
Key: YARN-2209
URL: https://issues.apache.org/jira/browse/YARN-2209
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2209.1.patch, YARN-2209.2.patch
YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate
application to re-register on RM restart. we should do the same for
AMS#allocate call also.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[
https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073999#comment-14073999
]
Hadoop QA commented on YARN-2209:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657746/YARN-2209.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.client.TestApplicationMasterServiceOnHA
org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4424//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4424//console
This message is automatically generated.
Replace allocate#resync command with ApplicationMasterNotRegisteredException
to indicate AM to re-register on RM restart
Key: YARN-2209
URL: https://issues.apache.org/jira/browse/YARN-2209
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2209.1.patch, YARN-2209.2.patch, YARN-2209.3.patch
YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate
application to re-register on RM restart. we should do the same for
AMS#allocate call also.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[
https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047402#comment-14047402
]
Hadoop QA commented on YARN-2209:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653095/YARN-2209.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1261 javac
compiler warnings (more than the trunk's current 1258 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4138//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/4138//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4138//console
This message is automatically generated.
Replace allocate#resync command with ApplicationMasterNotRegisteredException
to indicate AM to re-register on RM restart
Key: YARN-2209
URL: https://issues.apache.org/jira/browse/YARN-2209
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-2209.1.patch
YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate
application to re-register on RM restart. we should do the same for
AMS#allocate call also.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2209) Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart
[ https://issues.apache.org/jira/browse/YARN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044096#comment-14044096 ] Vinod Kumar Vavilapalli commented on YARN-2209: --- Makes sense. We need to be consistent across allocate/unregister calls. For things like shut-down and resync, I prefer exceptions. We can deprecate the corresponding AMCommands. Replace allocate#resync command with ApplicationMasterNotRegisteredException to indicate AM to re-register on RM restart Key: YARN-2209 URL: https://issues.apache.org/jira/browse/YARN-2209 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Jian He YARN-1365 introduced an ApplicationMasterNotRegisteredException to indicate application to re-register on RM restart. we should do the same for AMS#allocate call also. -- This message was sent by Atlassian JIRA (v6.2#6252)
