If you have a very heavy write load (like YCSB when only inserting), then you really have to tune HBase for that kind of workload since it's not the "normal" use case. Setting MAX_FILESIZE really high (1-2GB) and even pre-splitting the table when creating it (available in 0.89) will help.
Most of the time spent waiting is due to splitting and blocking due to either MemStores growing over their max size and the global MemStore size limit being reached. It's kinda rough and could probably be "smoother", but do you really have a use case that requires it or just poking? J-D On Thu, Sep 9, 2010 at 7:32 PM, Tao Xie <[email protected]> wrote: > hi, all > I use YCSB to measure the insert/read latency of hbase. > I found there will be 0 records inserted in up to 10 seconds during the > insertion procedure. > See the following result at 1514 second. I want to know why this occurs. Is > this due to compaction? > And I also want to know why the ops/sec varies all the time. Seems no a > stable time. > Thanks. > > 2010-09-10 00:07:29,608 1484 sec: 28786280 operations; 23475.3 current > ops/sec; [INSERT AverageLatency(ms)=8.81] > 2010-09-10 00:07:39,610 1494 sec: 28842632 operations; 5634.07 current > ops/sec; [INSERT AverageLatency(ms)=6.68] > 2010-09-10 00:07:49,612 1504 sec: 28964728 operations; 12207.16 current > ops/sec; [INSERT AverageLatency(ms)=7.68] > 2010-09-10 00:07:59,614 1514 sec: 28964728 operations; 0 current ops/sec; > 2010-09-10 00:08:10,778 1525 sec: 29130475 operations; 14846.56 current > ops/sec; [INSERT AverageLatency(ms)=24.45] > 2010-09-10 00:08:20,782 1535 sec: 29606967 operations; 47630.15 current > ops/sec; [INSERT AverageLatency(ms)=12.64] > 2010-09-10 00:08:30,784 1545 sec: 29908624 operations; 30159.67 current > ops/sec; [INSERT AverageLatency(ms)=0.12] > 2010-09-10 00:08:40,786 1555 sec: 30016632 operations; 10798.64 current > ops/sec; [INSERT AverageLatency(ms)=5.66] >
