Hi, we are using version 1.0.0+cdh5.4.4+160 We have heavy write load, ~ 10K per econd We have 10 nodes 7 disks each. I read some perf notes, they state that HBase can handle 1K per second writes per node without any problems.
I see some spikes on "writers". Write operation timing "jumps" from 40-50ms to 200-500ms Probably I hit memstore limit. RegionServer starts to flush memstore and stop to accept updates. I have several questions: 1. Does 4/(8 in hyperthreading) CPU + 7HDD node could absorb 1K writes per second? 2. What is the right way to fight with blocked writes? 2.1. What I did: hbase.hregion.memstore.flush.size to 256M to produce larger HFiles when flushing memstore base.hregion.memstore.block.multiplier to 4, since I have only one intensive-write table. Let it grow hbase.regionserver.optionallogflushinterval to 10s, i CAN loose some data, NP here. The idea that I reduce I/O pressure on disks. === Not sure if I can correctly play with these parameters. hbase.hstore.blockingStoreFiles=10 hbase.hstore.compactionThreshold=3
