[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13640329#comment-13640329 ] Hudson commented on YARN-561: - Integrated in Hadoop-Yarn-trunk #193 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/193/]) YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong. (Revision 1471156) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1471156 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/src/main/java/org/apache/hadoop/yarn/applications/unmanagedamlauncher/UnmanagedAMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationContainerInitEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/Container.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainersLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/ContainerInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLinuxContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLinuxContainerExecutorWithMocks.java *
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13640414#comment-13640414 ] Hudson commented on YARN-561: - Integrated in Hadoop-Hdfs-trunk #1382 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1382/]) YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong. (Revision 1471156) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1471156 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/src/main/java/org/apache/hadoop/yarn/applications/unmanagedamlauncher/UnmanagedAMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationContainerInitEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/Container.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainersLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/ContainerInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLinuxContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLinuxContainerExecutorWithMocks.java *
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639309#comment-13639309
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12580081/YARN-561.10.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/805//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/805//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.10.patch, YARN-561.1.patch, YARN-561.2.patch,
YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch,
YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639348#comment-13639348
]
Hitesh Shah commented on YARN-561:
--
Comments:
ApplicationAttemptId appAttemptIdEnv - please use a more appropriate variable
name.
System.getenv(Environment.CONTAINER_ID.name()) - what happens if the container
id is not set in the env?
{code}
putEnvIfNotNull(environment, Environment.USER.name(), container.getUser());
{code}
Is the user allowed to override the user env var?
{code}
+ LOG.info(Adding + container.getContainer().getId()
+ to application + app.toString());
{code}
- Doesn't the above happen so often that it will be a performance overhead?
Should this be changed to DEBUG? @Vinod, any comments on this?
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.10.patch, YARN-561.1.patch, YARN-561.2.patch,
YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch,
YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639476#comment-13639476 ] Vinod Kumar Vavilapalli commented on YARN-561: -- bq. ApplicationAttemptId appAttemptIdEnv - please use a more appropriate variable name. +1 bq. System.getenv(Environment.CONTAINER_ID.name()) - what happens if the container id is not set in the env? Unless you are looking to run new MR APP against old YARN, this shouldn't happen because NM always sets it now, no? bq. Is the user allowed to override the user env var? No. Mostly PWD too. But let's do those separately, this one dragged on for too long. bq. Doesn't the above happen so often that it will be a performance overhead? Should be fine for now, as there are other places like this. We should do a comprehensive log cleanup separately. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.10.patch, YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639514#comment-13639514 ] Vinod Kumar Vavilapalli commented on YARN-561: -- The latest patch also removed testing of Environment.MALLOC_ARENA_MAX, but I believe the current test isn't really the right way neither is it sufficient. We should really be validating YarnConfiguration.NM_ADMIN_USER_ENV. Can you please file a separate ticket for that? Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.10.patch, YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639568#comment-13639568 ] Hitesh Shah commented on YARN-561: -- @Vinod, there are also checks in the MRAppMaster which pretty much validate everything that the NM is supposed to be setting in the env. ( look for callers of MRAppMaster#validateInputParam ). This can be probably addressed separately in any case. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.10.patch, YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639624#comment-13639624
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12580122/YARN-561.11.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/807//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/807//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.10.patch, YARN-561.11.patch, YARN-561.1.patch,
YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch,
YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639640#comment-13639640 ] Vinod Kumar Vavilapalli commented on YARN-561: -- bq. there are also checks in the MRAppMaster which pretty much validate everything that the NM is supposed to be setting in the env. ( look for callers of MRAppMaster#validateInputParam ). This can be probably addressed separately in any case. Yeah, let's do those separately. The latest patch looks good, +1, checking it in. Xuan, can you please file all the follow up tickets? Thanks. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.10.patch, YARN-561.11.patch, YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639643#comment-13639643 ] Xuan Gong commented on YARN-561: Yes, I will do that. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.10.patch, YARN-561.11.patch, YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639673#comment-13639673 ] Vinod Kumar Vavilapalli commented on YARN-561: -- Xuan, we need a separate patch for branch-2, the trunk patch isn't applying. Can you please generate one, run all the tests on branch-2 applying that patch? We can't do that on Jenkins, as it isn't built for running tests on branches. Tx. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.10.patch, YARN-561.11.patch, YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639794#comment-13639794
]
Hadoop QA commented on YARN-561:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12580178/YARN-561.12.branch_2.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/812//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.10.patch, YARN-561.11.patch,
YARN-561.12.branch_2.patch, YARN-561.1.patch, YARN-561.2.patch,
YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch,
YARN-561.7.patch, YARN-561.8.patch, YARN-561.9.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638612#comment-13638612 ] Xuan Gong commented on YARN-561: Apply the latest comments: 1. Add Local_dirs as env for UnmanagedAMLauncher, set the value as /tmp 2. Remove setNewEnvironmentHack() from TestContainerLaunch. The function setNewEnvironmentHack() is setting some env variable, when CLC to call getenv(), those variables will go to MapString, String environment in CLC. Since I manually add environment for CLC, those will be duplicated. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638680#comment-13638680
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579946/YARN-561.9.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/801//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/801//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch,
YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch,
YARN-561.8.patch, YARN-561.9.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636124#comment-13636124
]
Hadoop QA commented on YARN-561:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579483/YARN-561.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainersMonitor
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/781//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/781//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch,
YARN-561.4.patch, YARN-561.5.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636149#comment-13636149
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579495/YARN-561.6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/782//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/782//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch,
YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636986#comment-13636986
]
Hadoop QA commented on YARN-561:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579626/YARN-561.7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/785//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch,
YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637012#comment-13637012 ] Xuan Gong commented on YARN-561: create the patch based on the latest trunk version Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch, YARN-561.8.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637030#comment-13637030
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579633/YARN-561.8.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/786//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/786//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch,
YARN-561.4.patch, YARN-561.5.patch, YARN-561.6.patch, YARN-561.7.patch,
YARN-561.8.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636093#comment-13636093 ] Xuan Gong commented on YARN-561: The new patch includes several changes: 1. move node_http_port to Environment 2. in order to set containerId, node_host, node_port and node_http_port at containerLaunch, we export org.apache.hadoop.yarn.api.records.Container into org.apache.hadoop.yarn.server.nodemanager.containermanager.container. 3. Remove containerId, resource from org.apache.hadoop.yarn.server.nodemanager.containermanager.container. We already export container API, we can get ContainerId, resource from there 4. Fix the related test cases changes. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636095#comment-13636095 ] Xuan Gong commented on YARN-561: Also remove APPLICATION_ATTEMPT_ID_ENV from MapReduceChildJVM Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch, YARN-561.5.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634404#comment-13634404 ] Xuan Gong commented on YARN-561: 1.Remove _ENV postfix from the new variables 2.Instead of using putEnvIfNotNull(), set variable value directly by using put() which These new envs can be enforced by NM 3.Modify the testcase, user provides these new envs as part of launch-context,, and validate that envs will get their value from NM. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634453#comment-13634453
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579188/YARN-561.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/764//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/764//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch,
YARN-561.4.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634552#comment-13634552 ] Vinod Kumar Vavilapalli commented on YARN-561: -- Good catch, Hitesh! Clearly none of these envs that NM will and should export will be needed to be exported by clients like AMLauncher/MRApp and may be distributed shell. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634660#comment-13634660 ] Xuan Gong commented on YARN-561: bq:Any reason why NM_PORT is set by NM but not NM_HTTP_PORT? Looks like we can remove NM_PORT and use NM_HTTP_PORT instead. bq:Why does AMLauncher need to set NM* related env vars? Originally, AMLauncher set those variable for the AM Container. Now we move all the variables to Environment for all the containers. So we do not need to set those variables anymore. At ContainerLaunch, these variables will be set by NM anyway. bq:Should MapReduceChildJVM need to continue to set APPLICATION_ATTEMPT_ID_ENV? I think we do not need to. The variable APPLICATION_ATTEMPT_ID_ENV is used in two places: a. At MapReduceChildJVM, it set APPLICATION_ATTEMPT_ID_ENV with value from APPLICATION_ATTEMPT_ID b. At YarnChild, it used APPLICATION_ATTEMPT_ID_ENV to set APPLICATION_ATTEMPT_ID Originally, we set APPLICATION_ATTEMPT_ID_ENV with APPLICATION_ATTEMPT_ID, and if we try to APPLICATION_ATTEMPT_ID in future, we just read the value from APPLICATION_ATTEMPT_ID_ENV. Now, we can get APPLICATION_ATTEMPT_ID directly from ContainerId, since we export those info. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634678#comment-13634678 ] Xuan Gong commented on YARN-561: We may need to keep both NM_PORT and NM_HTTP_PORT, for the MRAppMaster, it need both of them to do the initialization. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch, YARN-561.4.patch Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13633789#comment-13633789
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579088/YARN-561.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/757//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/757//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch, YARN-561.2.patch, YARN-561.3.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[
https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13631903#comment-13631903
]
Hadoop QA commented on YARN-561:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578747/YARN-561.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/738//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/738//console
This message is automatically generated.
Nodemanager should set some key information into the environment of every
container that it launches.
-
Key: YARN-561
URL: https://issues.apache.org/jira/browse/YARN-561
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Hitesh Shah
Assignee: Xuan Gong
Labels: usability
Attachments: YARN-561.1.patch
Information such as containerId, nodemanager hostname, nodemanager port is
not set in the environment when any container is launched.
For an AM, the RM does all of this for it but for a container launched by an
application, all of the above need to be set by the ApplicationMaster.
At the minimum, container id would be a useful piece of information. If the
container wishes to talk to its local NM, the nodemanager related information
would also come in handy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629837#comment-13629837 ] Xuan Gong commented on YARN-561: org.apache.hadoop.yarn.api.records.container has containerId and NodeId(which can get address and port) which are enough for container talked to its local NM. And by YARN-486, we have already add org.apache.hadoop.yarn.api.records.container to ContainImpl. So, it will get those information now. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630493#comment-13630493 ] Vinod Kumar Vavilapalli commented on YARN-561: -- Xuan, what Hitesh is saying is that when a container starts as a process, it doesn't know its containerId. We should make NM export it as part of the env. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630626#comment-13630626 ] Xuan Gong commented on YARN-561: when container starts as a process, it does not know its containerId. Does it mean when we execute the script to launch the container, the script does not include this containerId ? If I understand it correctly, we can solve this issue like this: 1. We need to add some content into enum Environment, such as ContainerId(String)(Which can be converted back by using ConverterUtils.toContainerId(String containerId)), NM hostName(String), NMPort(int). 2. The container Launch script is write out at ContainerLaunch::call(), and the environment is also set here. At ContainerLaunch, we already have org.apache.hadoop.yarn.server.nodemanager.containermanager.container, so containerId can be simply get. The NM hostName and NMPort can be get from NM_NodeId which is in NMContext. And ContainerLaunch is initialized from ContainerLauncher which already has NMContext. So, we can make changes here, when we initialize the ContainerLaunch, we either input NMContext as parameter, or simply give NM_NodeId, or just give NM_hostName and NMPort, then we can get all the information we need. Any other suggestions ?? Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630635#comment-13630635 ] Hitesh Shah commented on YARN-561: -- @Xuan, one thing to be careful of this certain env settings should only be set by the NodeManager when it launches the container and not by an application. So you would need a notion of certain whitelist environment variables that should be set only by the NM and not overridden by the env in CLC provided by the application. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630652#comment-13630652 ] Xuan Gong commented on YARN-561: Just like ApplicationConstants which include some variables can only set in the AppMaster environment ? At the beginning (From the code ContainerLaunch::call()), the env is original from CLC.getEnvironment(), then we can set ContainerId, and Node_hostName, Node_portNumber after that. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630681#comment-13630681 ] Hitesh Shah commented on YARN-561: -- Take a look at ContainerLaunch#sanitizeEnv() and how it handles non-modifiable environment variables. The above mentioned env variables should also fall into the non-modifiable category. Nodemanager should set some key information into the environment of every container that it launches. - Key: YARN-561 URL: https://issues.apache.org/jira/browse/YARN-561 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah Assignee: Xuan Gong Labels: usability Information such as containerId, nodemanager hostname, nodemanager port is not set in the environment when any container is launched. For an AM, the RM does all of this for it but for a container launched by an application, all of the above need to be set by the ApplicationMaster. At the minimum, container id would be a useful piece of information. If the container wishes to talk to its local NM, the nodemanager related information would also come in handy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
