[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684210#comment-13684210 ] Hudson commented on YARN-639: - Integrated in Hadoop-Mapreduce-trunk #1458 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1458/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684184#comment-13684184 ] Hudson commented on YARN-639: - Integrated in Hadoop-Hdfs-trunk #1431 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1431/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684146#comment-13684146 ] Hudson commented on YARN-639: - Integrated in Hadoop-Yarn-trunk #241 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/241/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683918#comment-13683918 ] Hudson commented on YARN-639: - Integrated in Hadoop-trunk-Commit #3930 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3930/]) YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. (Revision 1493280) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493280 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683909#comment-13683909 ] Vinod Kumar Vavilapalli commented on YARN-639: -- +1. Looks good. Checking this in. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[
https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683874#comment-13683874
]
Hadoop QA commented on YARN-639:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587912/YARN-639.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1252//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1252//console
This message is automatically generated.
> Make AM of Distributed Shell Use NMClient
> -
>
> Key: YARN-639
> URL: https://issues.apache.org/jira/browse/YARN-639
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-639.1.patch, YARN-639.2.patch, YARN-639.2.patch
>
>
> YARN-422 adds NMClient. AM of Distributed Shell should use it instead of
> using ContainerManager directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[ https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683781#comment-13683781 ] Zhijie Shen commented on YARN-639: -- bq. why do we need to do getContainerStatus after successfully starting it? is it required? it's not required for functioning of the distributed shell. However, since distributed shell somehow servers as a demo application, I'd like to have getContainerStatus there to show the usage of NMClientAsync and its callback handlers. > Make AM of Distributed Shell Use NMClient > - > > Key: YARN-639 > URL: https://issues.apache.org/jira/browse/YARN-639 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-639.1.patch > > > YARN-422 adds NMClient. AM of Distributed Shell should use it instead of > using ContainerManager directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[
https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682914#comment-13682914
]
Omkar Vinit Joshi commented on YARN-639:
[~zjshen] looked at your patch... minor comments
bq. +nodeManager.stop();
* in AM, can you rename nodemanager to something like nmClient? It was
confusing when I saw nodemanager.stop() :).
{code}
+@Override
+public void onContainerStarted(ContainerId containerId,
+Map allServiceResponse) {
+ LOG.info("Succeeded to start Container " + containerId);
+ Container container = containers.get(containerId);
+ if (container != null) {
+nodeManager.getContainerStatus(containerId, container.getNodeId(),
+container.getContainerToken());
+ }
+}
{code}
* why do we need to do getContainerStatus after successfully starting it? is it
required?
* also can you change LOG.info in CallBackHandler to LOG.debug / LOG.error as
appropriate?
everything else looks good..
> Make AM of Distributed Shell Use NMClient
> -
>
> Key: YARN-639
> URL: https://issues.apache.org/jira/browse/YARN-639
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-639.1.patch
>
>
> YARN-422 adds NMClient. AM of Distributed Shell should use it instead of
> using ContainerManager directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-639) Make AM of Distributed Shell Use NMClient
[
https://issues.apache.org/jira/browse/YARN-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669514#comment-13669514
]
Hadoop QA commented on YARN-639:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585253/YARN-639.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1025//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1025//console
This message is automatically generated.
> Make AM of Distributed Shell Use NMClient
> -
>
> Key: YARN-639
> URL: https://issues.apache.org/jira/browse/YARN-639
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-639.1.patch
>
>
> YARN-422 adds NMClient. AM of Distributed Shell should use it instead of
> using ContainerManager directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
