[
https://issues.apache.org/jira/browse/YARN-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15679132#comment-15679132
]
Varun Saxena edited comment on YARN-5877 at 11/19/16 11:55 AM:
---------------------------------------------------------------
Is this the right thing to do ?
NM whitelist configuration is used to allow containers to override NM's default
environment.
Let us assume there is an environment variable DUMMY_ENV. Assume in some
deployment, for some jobs, its set in the ContainerLaunchContext and for all
other jobs we want a default which can be set in NM environment.
Before the change in the patch, we could have set DUMMY_ENV in NM whitelist and
NM would have picked up value for this environment variable from container
launch context (if available) otherwise taken it from NM environment.
After the change in this patch, NM would keep the value coming from container
launch context as default and use that only if the environment variable on NM
is not set.
This is a departure from current behavior. Probably you can argue none of us
are using it this way but its still a change in behavior.
We have done the same for HADOOP_YARN_HOME, HADOOP_COMMON_HOME, etc. in
YARN-3853 but these are well known environment variables so probably nobody
uses them the way I mentioned above. Atleast we do not. So it may be safe to
use them this way.
If we were to have a different configuration, we can probably get rid of these
concerns.
Thoughts ?
was (Author: varun_saxena):
Is this the right thing to do ?
NM whitelist configuration is used to allow containers to override NM's default
environment.
Let us assume there is an environment variable DUMMY_ENV. Assume in some
deployment, for some jobs, its set in the ContainerLaunchContext and for all
other jobs we want a default which can be set in NM environment.
Before the change in the patch, we could have set DUMMY_ENV in NM whitelist and
NM would have picked up value for this environment variable from container
launch context (if available) otherwise taken it from NM environment.
After the change in this patch, NM would keep the value coming from container
launch context as default and use that only if the environment variable on NM
is not set.
This is a departure from current behavior. Probably you can argue none of us
are using it this way but its still a change in behavior.
We have done the same for HADOOP_YARN_HOME, HADOOP_COMMON_HOME, etc. in
YARN-3853 but these are well known environment variables so probably nobody
uses them the way I mentioned above. So its safe to use them this way. If we
were to have a different configuration, we can probably get rid of these
concerns.
Thoughts ?
> Allow all nm-whitelist-env to get overridden during launch
> ----------------------------------------------------------
>
> Key: YARN-5877
> URL: https://issues.apache.org/jira/browse/YARN-5877
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
> Attachments: Dockerfile, YARN-5877.0001.patch, YARN-5877.0002.patch,
> bootstrap.sh, yarn-site.xml
>
>
> As per the {{yarn.nodemanager.env-whitelist}} for the configured values
> should containers may override rather than use NodeManager's default.
> {code}
> <property>
> <description>Environment variables that containers may override rather
> than use NodeManager's default.</description>
> <name>yarn.nodemanager.env-whitelist</name>
>
> <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME</value>
> </property>
> {code}
> But only the following containers can override
> {code}
> whitelist.add(ApplicationConstants.Environment.HADOOP_YARN_HOME.name());
> whitelist.add(ApplicationConstants.Environment.HADOOP_COMMON_HOME.name());
> whitelist.add(ApplicationConstants.Environment.HADOOP_HDFS_HOME.name());
> whitelist.add(ApplicationConstants.Environment.HADOOP_CONF_DIR.name());
> whitelist.add(ApplicationConstants.Environment.JAVA_HOME.name());
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
