While working on HBASE-3894 I was wondering what the best way to test performance sensitive patches to HBase was. There is the o.a.h.h.PerformanceEvaluation class, but it doesn't appear have been really touched in close to a year, and I haven't heard much mention of it on the mailing lists in awhile. (The numbers at http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation are even further out of date). Does anyone have a good sense of the usefulness of this tool?
There's also YCSB, which I've seen more people mention recently. However, the book says "TODO: Describe how YCSB is poor for putting up a decent cluster load." ( http://hbase.apache.org/book.html#d470e4848 ) so I get the sense that it's not a great test for HBase. Are there any others? Does anyone have thoughts about the best way to run hbase through some benchmarks to evaluate how patches may affect performance? Would it be worth trying to build a new set of benchmarks for hbase, or taking one of these existing ones and improving it some way? Dave
