[
https://issues.apache.org/jira/browse/HADOOP-11022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114606#comment-14114606
]
Allen Wittenauer edited comment on HADOOP-11022 at 8/28/14 11:45 PM:
---------------------------------------------------------------------
The basic problem is the export of HADOOP_ENV_PROCESSED. This is used to
prevent hadoop-env.sh from being re-read. This protection makes sense if you
have, e.g., FOO_OPT="$FOO_OPTS blah" in this file. But with the curren state
of the world, there's quite a few things going wrong here:
1. We don't protect the other *-env.sh files like we do hadoop-env.sh. i.e.,
yarn-env.sh will still get re-read multiple times.
2. This only works correctly because the vars in hadoop-env.sh are also
exported.... which is probably the wrong thing to do as well.
3. Now that there are more than just vars in *-env.sh, the system really does
need to re-read *-env.sh to pick those up.
Some potential solutions worth exploring:
* What happens if we no longer export anything in *-env.sh and re-read them at
every command run like we do with the other files? (This brings Hadoop in line
with most other OS utilities.) One side effect: users upgrading to trunk would
need to also remove the export lines as part of upgrading.
* If we protect all the *-env.sh and move user defined functions to a new file?
This has a lot of potential! One big problem is what to do about YARN_OPTS.
was (Author: aw):
The basic problem is the export of HADOOP_ENV_PROCESSED. This is used to
prevent hadoop-env.sh from being re-read. This protection makes sense if you
have, e.g., FOO_OPT="${FOO_OPTS} blah" in this file. But with the curren state
of the world, there's quite a few things going wrong here:
1. We don't protect the other *-env.sh files like we do hadoop-env.sh. i.e.,
yarn-env.sh will still get re-read multiple times.
2. This only works correctly because the vars in hadoop-env.sh are also
exported.... which is probably the wrong thing to do as well.
3. Now that there are more than just vars in *-env.sh, the system really does
need to re-read *-env.sh to pick those up.
Some potential solutions worth exploring:
* What happens if we no longer export anything in *-env.sh and re-read them at
every command run like we do with the other files? (This brings Hadoop in line
with most other OS utilities.) One side effect: users upgrading to trunk would
need to also remove the export lines as part of upgrading.
* If we protect all the *-env.sh and move user defined functions to a new file?
This has a lot of potential! One big problem is what to do about YARN_OPTS.
> User replaced functions get lost 2-3 levels deep (e.g., sbin)
> -------------------------------------------------------------
>
> Key: HADOOP-11022
> URL: https://issues.apache.org/jira/browse/HADOOP-11022
> Project: Hadoop Common
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Priority: Critical
>
> The code that protects hadoop-env.sh from being re-executed is also causing
> functions that the user replaced to get overridden with the defaults. This
> typically happens when running commands that nest, such as most of the
> content in sbin. Just running stuff out of bin (e.g., bin/hdfs --daemon
> start namenode) does not trigger this behavior.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
