Hi James, In my 10 nodes cluster, it used to take 7 minutes (3 minutes M/R + 4 minutes load to mysql) to process data and being able to visualize on HICC UI. Now, it takes 50 milliseconds. For data aggregation, it used to take 15-20 minutes to roll up data for 2000 nodes data daily, now it takes <5 minutes. The improvement is 2100 times better for data load latency, and 3 times better for data analytics throughput with pig+hbase.
regards, Eric On Sat, Nov 20, 2010 at 12:20 PM, James Seigel <[email protected]> wrote: > Hello! > > As a high volume user, I was just wondering how the HbaseWriter compares with > the current one under load? Better or worse and by how much? > > Cheers > James. > > > On 2010-11-20, at 1:15 PM, Eric Yang wrote: > >> Hi all, >> >> In order to use full features of Chukwa in trunk, HBase is required to >> display data on HICC. I am wondering if anyone has good success in >> using HBase+HICC? I am leaning toward making hbase the default data >> storage for chukwa, and the default configuration for chukwa collector >> will make use of HBaseWriter. What do the community feel about >> changing the default writer config? >> >> regards, >> Eric > >
