Terence Yim created YARN-4727:
---------------------------------
Summary: Unable to override the $HADOOP_CONF_DIR env variable for
container
Key: YARN-4727
URL: https://issues.apache.org/jira/browse/YARN-4727
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.6.4, 2.7.2, 2.5.2, 2.4.1
Reporter: Terence Yim
Given the default config of "yarn.nodemanager.env-whitelist", application
should be able to set the env variable $HADOOP_CONF_DIR to value other than the
one in the NodeManager system environment. However, I believe due to a bug in
the
{{org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch}}
class, it is not possible so.
>From the {{sanitizeEnv()}} method in the ContainerLaunch class
>(https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java#L991)
{noformat}
String[] whitelist = conf.get(YarnConfiguration.NM_ENV_WHITELIST,
YarnConfiguration.DEFAULT_NM_ENV_WHITELIST).split(",");
for(String whitelistEnvVariable : whitelist) {
putEnvIfAbsent(environment, whitelistEnvVariable.trim());
}
...
private static void putEnvIfAbsent(
Map<String, String> environment, String variable) {
if (environment.get(variable) == null) {
putEnvIfNotNull(environment, variable, System.getenv(variable));
}
}
{noformat}
The {{putEnvIfAbsent}} call simply use the {{whitelistEnvVariable}} to call
{{System.getenv(whitelistEnvVariable)}} and set it to the environment map. It
should be using the one from the {{launchContext}} instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
