[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684210#comment-13684210 ] Hudson commented on YARN-639: - Integrated in Hadoop-Mapreduce-trunk #1458 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1458/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684184#comment-13684184 ] Hudson commented on YARN-639: - Integrated in Hadoop-Hdfs-trunk #1431 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1431/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684146#comment-13684146 ] Hudson commented on YARN-639: - Integrated in Hadoop-Yarn-trunk #241 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/241/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683918#comment-13683918 ] Hudson commented on YARN-639: - Integrated in Hadoop-trunk-Commit #3930 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3930/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683909#comment-13683909 ] Vinod Kumar Vavilapalli commented on YARN-639: -- +1. Looks good. Checking this in. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683874#comment-13683874 ] Hadoop QA commented on YARN-639: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12587912/YARN-639.2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1252//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1252//console This message is automatically generated. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683781#comment-13683781 ] Zhijie Shen commented on YARN-639: -- bq. why do we need to do getContainerStatus after successfully starting it? is it required? it's not required for functioning of the distributed shell. However, since distributed shell somehow servers as a demo application, I'd like to have getContainerStatus there to show the usage of NMClientAsync and its callback handlers. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682914#comment-13682914 ] Omkar Vinit Joshi commented on YARN-639: [~zjshen] looked at your patch... minor comments bq. +nodeManager.stop(); * in AM, can you rename nodemanager to something like nmClient? It was confusing when I saw nodemanager.stop() :). {code} +@Override +public void onContainerStarted(ContainerId containerId, +Map allServiceResponse) { + LOG.info("Succeeded to start Container " + containerId); + Container container = containers.get(containerId); + if (container != null) { +nodeManager.getContainerStatus(containerId, container.getNodeId(), +container.getContainerToken()); + } +} {code} * why do we need to do getContainerStatus after successfully starting it? is it required? * also can you change LOG.info in CallBackHandler to LOG.debug / LOG.error as appropriate? everything else looks good.. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669514#comment-13669514 ] Hadoop QA commented on YARN-639: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585253/YARN-639.1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1025//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1025//console This message is automatically generated. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira