This isn't entirely true. If you look at agent.sh, it uses HADOOP_20_JAR, only. Which is hardcoded in bin/chukwa-config.sh. That script also sets HADOOP_18_JAR, which as near as I can tell is used nowhere.
Also. Can we document some of our Hadoop version discoveries? It's not at all obvious from the current admin guide that the processing cluster needs to be running 0.18, and that the cluster being monitored ought to be 0.20. --Ari On Wed, Apr 29, 2009 at 10:19 AM, Eric Yang <[email protected]> wrote: > Hi Ari, > > Hadoop jars should be autodetected from either HADOOP_HOME, or hadoopjars > directory. For the agent side, you could remove the hadoop-0.18.3.jar from > the hadoopjars directory to make data collection work with new version of > hadoop. The current stopgap is messy because demux depends on hadoop-0.18.3 > jar file. It was not the best approach to place hadoop-0.18.3 jar file in > hadoopjars directory. > > Short term, use HADOOP_HOME to control location of HADOOP jar file. > Mid term, create individual RPM file which bundles the right hadoop jars. > Long term, Upgrade demux pipeline to use hadoop 0.20.0+. > > Regards, > Eric > > On 4/28/09 10:34 PM, "Ariel Rabkin" <[email protected]> wrote: > >> Hi all. >> >> So it looks like we're now hardcoding which hadoop jar to use in >> chukwa-config.sh -- we always use the ones in hadoopjars in preference >> to trying to autodetect, and we hardcode the versions we want. Is this >> a stopgap, or a permanent measure? >> >> --Ari > > -- Ari Rabkin [email protected] UC Berkeley Computer Science Department
