[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176961#comment-16176961
]
Eric Badger commented on YARN-7226:
-----------------------------------
I can see the need for Docker containers to supply their own env vars in the
image, but I agree that it should not change previous behavior of the
whitelist. The client gets to decide what image to use, so they should know the
layout of the image anyway. And if they don't, then they can fall back to the
environment of the docker image. If we want for there to be a separate list of
env vars that cannot be changed, then we can do that, but I think it should be
empty and/or disabled by default to preserve current behavior. Otherwise, as
[~jlowe] said, users won't be able to supply their own values for the env vars
in the whitelist. So my vote would be for putting this patch in and then filing
a followup jira to add a NM configurable list of variables that cannot be
changed by the user (empty/disabled by default)
> Whitelisted variables do not support delayed variable expansion
> ---------------------------------------------------------------
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
