I participated on the HADOOP-9450 code review. There was some debate about the significance of keeping HADOOP_CONF_DIR first. Ultimately, there was agreement that "FIRST" really ought to mean first. Apparently, other Hadoop ecosystem projects have made the same interpretation and implemented their scripts accordingly. Just echoing what Allen said, there were always plenty of ways for a user to override *-site.xml too.
Of course, I'm sorry to hear the change caused an issue at your company, Sangjin. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Sep 10, 2014 at 10:55 AM, Allen Wittenauer <[email protected]> wrote: > > On Sep 10, 2014, at 10:14 AM, Sangjin Lee <[email protected]> wrote: > > > if HADOOP_USER_CLASSPATH_FIRST is > > set and the user provides his/her version of *-site.xml through > > HADOOP_CLASSPATH, the user would end up trumping the hadoop > configuration. > > And I believe this behavior is preserved after Allen's changes > > (HADOOP-9902). > > > > I'd be surprised if 9902 changed this behavior, especially given > that HADOOP_CONF_DIR is added fairly quickly to the CLASSPATH and > HADOOP_USER_CLASSPATH is processed last or near last. > > > Is this an intended behavior? What I'm not sure of is whether we expect > the > > client to be able to override the site.xml files the hadoop configuration > > provides. If that is true, then it is working as desired. If not, we'd > need > > to fix this behavior. > > > > Thoughts? > > > > Users have always been able to override the *-site.xml files via > java properties, jobconf/etc, pointing HADOOP_CONF_DIR to somewhere else, > or providing --conf flags to shell commands. In other words, this isn't > new behavior, but provides yet another option to override the provided > settings. > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
