Hi all,
We experienced a downtime in our HBase installation today. We have our HBase hosted in EC2 with 1 master (with ZK) and 3 slaves (all of them are m1.large). We are using HBase version 0.20.4 We have a method that opens a scanner and retrieves some values. The method approximately scans 300 rows. Each row has three column families and approximately 75 longs. The table is fairly small and in total the table has approximately 1500 rows. We tried calling this method under huge traffic and the CPU of the regionserver that hosted this table spiked to 100%. It brought down our application. We are doing multiple incrementColumnValue calls for every request for this traffic and HBase seems to take it well. So, does that mean, it's a bad idea to call a scanner under huge traffic? Will this problem get solved if we make a new table and store the values with a different key so that the exact value can be retrieved (with a Get call) ? Are there any other ways to resolve this hotspotting without duplicating data? Are we limited by what a machine can handle if we have a fairly small table (that can fit in a region server or possibly in a single region)? Are there any creative solutions people are using? Regards, Vaibhav Puranik GumGum
