Re: should clients be able to override *-site.xml?

2014-09-11 Thread Sangjin Lee
Thanks for the clarification. Yes I agree that there are many ways for users to override *-site.xml regardless of what the hadoop scripts do. It was more of a question of the interpretation of intent, and I got the answer. Thanks! Sangjin On Wed, Sep 10, 2014 at 4:34 PM, Chris Nauroth

should clients be able to override *-site.xml?

2014-09-10 Thread Sangjin Lee
tl;dr This was borne out of an investigation of a hadoop issue at our company. It appears that prior to https://issues.apache.org/jira/browse/HADOOP-9450 the hadoop configuration directory (HADOOP_CONF_DIR) was always the first entry in the classpath, regardless of HADOOP_USER_CLASSPATH_FIRST or

Re: should clients be able to override *-site.xml?

2014-09-10 Thread Allen Wittenauer
On Sep 10, 2014, at 10:14 AM, Sangjin Lee sj...@apache.org wrote: if HADOOP_USER_CLASSPATH_FIRST is set and the user provides his/her version of *-site.xml through HADOOP_CLASSPATH, the user would end up trumping the hadoop configuration. And I believe this behavior is preserved after

Re: should clients be able to override *-site.xml?

2014-09-10 Thread Chris Nauroth
I participated on the HADOOP-9450 code review. There was some debate about the significance of keeping HADOOP_CONF_DIR first. Ultimately, there was agreement that FIRST really ought to mean first. Apparently, other Hadoop ecosystem projects have made the same interpretation and implemented