The query is all random read. The scenario is that a user want to query his own monthly bill report, e.g. to query what happened on his bill in March, or Feb, etc. Since every user may want to do so, we can't predict who will be the next to ask for such monthly bill report.
2011/4/26 Stack <[email protected]>: >> Currently, to store bill records, we can achieve about 30K record/second. >> > > Can you use bulk load? See http://hbase.apache.org/bulk-loads.html > >> However, the query performance is quite poor. We can only achieve >> about 600~700 month_report/second. That is, each region server can >> only serve query for about 100 row/second. Block cache hit ratio is >> about 20%. >> > > This is random accesses? Why random accesses and not scans? > > >> Do you have any advice on how to improve the query performance? >> > > See above cited performance section from website book. > > >> Below is some metrics info reported by region server: >> 2011-04-26T10:56:12 hbase.regionserver: >> RegionServer=regionserver50820, blockCacheCount=40969, >> blockCacheEvictedCount=216359, blockCacheFree=671152504, >> blockCacheHitCachingRatio=20, blockCacheHitCount=67936, >> blockCacheHitRatio=20, blockCacheMissCount=257675, >> blockCacheSize=2743351688, compactionQueueSize=0, >> compactionSize_avg_time=0, compactionSize_num_ops=7, >> compactionTime_avg_time=0, compactionTime_num_ops=7, flushQueueSize=0, >> flushSize_avg_time=0, flushSize_num_ops=0, flushTime_avg_time=0, >> flushTime_num_ops=0, fsReadLatency_avg_time=46, >> fsReadLatency_num_ops=257905, fsSyncLatency_avg_time=0, >> fsSyncLatency_num_ops=1726, fsWriteLatency_avg_time=0, >> fsWriteLatency_num_ops=0, memstoreSizeMB=0, regions=169, >> requests=82.1, storefileIndexSizeMB=188, storefiles=343, stores=169 >> 2011-04-26T10:56:22 hbase.regionserver: >> RegionServer=regionserver50820, blockCacheCount=42500, >> blockCacheEvictedCount=216359, blockCacheFree=569659040, >> blockCacheHitCachingRatio=20, blockCacheHitCount=68418, >> blockCacheHitRatio=20, blockCacheMissCount=259206, >> blockCacheSize=2844845152, compactionQueueSize=0, >> compactionSize_avg_time=0, compactionSize_num_ops=7, >> compactionTime_avg_time=0, compactionTime_num_ops=7, flushQueueSize=0, >> flushSize_avg_time=0, flushSize_num_ops=0, flushTime_avg_time=0, >> flushTime_num_ops=0, fsReadLatency_avg_time=44, >> fsReadLatency_num_ops=259547, fsSyncLatency_avg_time=0, >> fsSyncLatency_num_ops=1736, fsWriteLatency_avg_time=0, >> fsWriteLatency_num_ops=0, memstoreSizeMB=0, regions=169, >> requests=92.2, storefileIndexSizeMB=188, storefiles=343, stores=169 >> 2011-04-26T10:56:32 hbase.regionserver: >> RegionServer=regionserver50820, blockCacheCount=39238, >> blockCacheEvictedCount=221509, blockCacheFree=785944072, >> blockCacheHitCachingRatio=20, blockCacheHitCount=69043, >> blockCacheHitRatio=20, blockCacheMissCount=261095, >> blockCacheSize=2628560120, compactionQueueSize=0, >> compactionSize_avg_time=0, compactionSize_num_ops=7, >> compactionTime_avg_time=0, compactionTime_num_ops=7, flushQueueSize=0, >> flushSize_avg_time=0, flushSize_num_ops=0, flushTime_avg_time=0, >> flushTime_num_ops=0, fsReadLatency_avg_time=39, >> fsReadLatency_num_ops=261070, fsSyncLatency_avg_time=0, >> fsSyncLatency_num_ops=1746, fsWriteLatency_avg_time=0, >> fsWriteLatency_num_ops=0, memstoreSizeMB=0, regions=169, >> requests=128.77777, storefileIndexSizeMB=188, storefiles=343, >> stores=169 >> > > This is hard to read but I don't see anything obnoxious. > > >> >> And we also tried to disable block cache, it seems the performance is >> even a little bit better. And it we use the configuration 6 DN servers >> + 3 RS servers, we can get better throughput at about 1000 >> month_report/second. I am confused. Can any one explain the reason? >> > > Sounds like you are doing all random reads? Do you have to? > > St.Ack >
