On 2016-07-29 03:05 (-0700), Ewan Higgs <ewan.hi...@hgst.com> wrote: > If HADOOP_CLASSPATH=$(hadoop classpath):/path/to/hadoop/tools/jar/dir then > hdfs gets very confused and somehow decides it wonât respect the > HADOOP_CONF_DIR. > > Solution: only add /path/to/hadoop/tools/jar/dir to HADOOP_CLASSPATH.
Short: HADOOP_CLASSPATH is supplemental/additional. The content of 'hadoop classpath' is always present. See the UnixShellGuide and hadoop-env.sh for more information, especially if you are upgrading or accustomed to previous releases. Long: HADOOP_CONF_DIR is generally pushed onto the actual classpath first unless other directives have been given elsewhere. Since you're using trunk, you've got some additional tools to figure out what is going on: $ HADOOP_CLASSPATH=$(hadoop classpath):/tmp hdfs --debug version This will spill out a bunch of info. Key messages will be the system trying to de-dupe the classpath (likely unsuccessfully due to shell expansion). But the one you care most about will be near the end. You'll get a line that says: DEBUG: Final CLASSPATH: The key thing there is that, at least in my test with a fresh install with the above command line, the very first entry is my HADOOP_CONF_DIR. So I'm not sure why HADOOP_CONF_DIR is getting ignored in your case. There might be other directives configured somewhere else that may be causing the system confusion or who knows what all since the jars themselves include *-default.xml to acts as the defaults. Additionally, since you mentioned /path/to/hadoop/tools/jar/dir, if this is the actual tools that ship with Hadoop, be aware that there are now much easier ways to add bits. You just need to edit the HADOOP_OPTIONAL_TOOLS line in hadoop-env.sh to include the functionality you want. No need to even modify HADOOP_CLASSPATH anymore to add, e.g., S3 or Azure. --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
