[
https://issues.apache.org/jira/browse/MAPREDUCE-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942751#comment-14942751
]
Allen Wittenauer commented on MAPREDUCE-6491:
---------------------------------------------
bq. do you think it should not be appended to by default?
For trunk, it should not. It is not guaranteed to work. If a user puts in
HADOOP_CLASSPATH="foo" in hadoop-env.sh (or .hadooprc, but that would be weird
on a compute node) the shell commands will NOT honor the append. (So, for
example, a user runs a streaming job that uses hadoop fs to add/remove files
from a bundled-with-the-job fs impl will fail.)
bq. If that's the case, then we should be changing that test case for
TestMRApps#testSetClasspathWithArchives to not rely on the appending behavior.
I'll need to look at it, but probably. It's probably working because
HADOOP_CLASSPATH isn't set at all in default, out-of-the-box configuration. On
real systems, this has a chance to break.
bq. Are you saying we should be adding HADOOP_TASK_CLASSPATH as an appendable
variable by default and take out HADOOP_CLASSPATH from the list?
We accidentally dropped from the hadoop-env.sh comments the description of
HADOOP_CLASSPATH, but it's effectively the user interface to setting additional
classpaths. So what might make sense is to keep HADOOP_CLASSPATH for users
here but it gets turned into HADOOP_TASK_CLASSPATH, which is meant for the app
framework to work with. Thoughts?
Of course, all of this is dependent upon MAPREDUCE-6458 getting finished too.
> Environment variable handling assumes values should be appended
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-6491
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6491
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Jason Lowe
> Assignee: Dustin Cote
> Attachments: YARN-2369-1.patch, YARN-2369-10.patch,
> YARN-2369-2.patch, YARN-2369-3.patch, YARN-2369-4.patch, YARN-2369-5.patch,
> YARN-2369-6.patch, YARN-2369-7.patch, YARN-2369-8.patch, YARN-2369-9.patch
>
>
> When processing environment variables for a container context the code
> assumes that the value should be appended to any pre-existing value in the
> environment. This may be desired behavior for handling path-like environment
> variables such as PATH, LD_LIBRARY_PATH, CLASSPATH, etc. but it is a
> non-intuitive and harmful way to handle any variable that does not have
> path-like semantics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
