Hi, All HBase could block the online write operations when there are too many data in memstore (to be more efficient for the potential compaction incurred by this flush when there're many files on disk). This blocking effect is also observed by others (e.g., http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html).
The solution come up with on the above web blog is to increase the Memstore size with fewer # of flushes, and to tolerate bigger # of files on disk (by increasing blockingStoreFiles). This is a kind of HBase tuning towards write intensive workload. My targeted application has dynamical workload which may changes from write-intensive to read-intensive. Also there are peak hours (when blocking is user perceivable and should not be invoked) and offpeak hours (when blocking is tolerable). I am wondering if there is any more intelligent solution (say a clever scheduling policy that blocks only at offpeak hours) exist in the latest HBase version that could minimizes the effect of write stream block? Regards Yun* *
