[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175635#comment-16175635
]
Sidharta Seethana commented on YARN-7226:
-----------------------------------------
{quote}
As I understand it, a whitelisted variable is simply an environment variable
that will be propagated from the nodemanager's environment to the container's
environment unless the container has specified it's own value for that variable.
{quote}
[~jlowe],
Using docker containers introduces a new way in which environment variables for
a container can be specified - this is through the docker image itself, which
is a fairly common scenario. For example, an image used for running MR tasks
could specify its own JAVA_HOME that needs to be used instead of the JAVA_HOME
that is specified in the nodemanager's environment. Since
{{launch_container.sh}} runs inside the docker container, using the specified
docker image, using {{putEnvIfAbsent}} for whitelisted env vars doesn't do the
right thing in this case.
> Whitelisted variables do not support delayed variable expansion
> ---------------------------------------------------------------
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
