It appears that the classpath was incorrect during the test runs where hadoop-0.16.2 had failures.
With the classpath corrected, hadoop-0.16.2 passed the HBase stress test 5 times out of 5. +1 for hadoop-0.16.2 ------------- About the HBase stress test: The HBase stress test puts both Hadoop and HBase under heavy load, which is why we use it to qualify HBase releases. In this test we run a 4 node cluster. All 4 nodes are data nodes. Node xxx-xxx-12.u.powerset.com runs the name node, JobTracker and HBase master. The test is also launched from this node. All 4 nodes run task trackers. The sole region server runs on xxx-xxx-13.u.powerset.com The application (PerformanceEvaluation) is launched from xxx-xxx-12.u.powerset.com but runs map jobs on all 4 nodes of the cluster. We install everything on -12 and use rsync to make sure all the cluster nodes are doing the same thing. Attachments: env.sh sets up the directories we want to rsync/run from. rsync.sh is how we push the software around the cluster. start.sh is how we start up hadoop and hbase. The command line for running PerformanceEvaluation is: $ date | tee start.time ; hbase-0.1.0/bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 4 ; date | tee end.time hadoop-env.sh - how we point hadoop at hbase hadoop-site.xml - just changes port numbers and hadoop.tmp.dir masters - points to xxx-xxx-12.u.powerset.com slaves - points to xxx-xxx-1[2-5].u.powerset.com For HBase: hbase-site.xml changes port numbers regionservers - points to xxx-xxx-13.u.powerset.com hbase-env.sh - points to JAVA_HOME and HBASE-HOME (only modifications from distribution) The key seems to be in $HBASE_HOME/bin/hbase - This file includes all jars in $HBASE_HOME/lib - because that directory contains hadoop jars, they get put on the classpath which would explain the strange behavior and mis-matched line numbers. hbase-0.1.0 ships with jars from hadoop-0.16.0. Replacing the 0.16.0 jars in HBASE_HOME/lib with the jars from hadoop-0.16.2 seems to have solved the problem. My bad. --- Jim Kellerman, Senior Engineer; Powerset No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.22.1/1352 - Release Date: 3/31/2008 10:13 AM
