Thank you Ted, It is a great explanation. You are always very helpful ^_^ I will study the link carefully.
Thanks, Ming -----Original Message----- From: Ted Yu [mailto:[email protected]] Sent: Friday, November 21, 2014 1:32 AM To: [email protected] Subject: Re: how to explain read/write performance change after modifying the hfile.block.cache.size? When block cache size increases from 0 to 0.4, the amount of heap given to memstore decreases. This would slow down the writes. Please see: http://hbase.apache.org/book.html#store.memstore For your second question, see this thread: http://search-hadoop.com/m/DHED4TEvBy1/lars+hbase+hflush&subj=Re+Clarifications+on+HBase+Durability Cheers On Thu, Nov 20, 2014 at 8:05 AM, Liu, Ming (HPIT-GADSC) <[email protected]> wrote: > Hello, all, > > I am playing with YCSB to test HBase performance. I am using HBase 0.98.5. > I tried to adjust the hfile.block.cache.size to see the difference, > when I set hfile.block.cache.size to 0, read performance is very bad, > but write performance is very very very good....; when I set > hfile.block.cache.size to 0.4, read is better, but write performance > drop dramatically. I disable the client side writebuffer already. > This is hard to understand for me: > The HBase guide just said hfile.block.cache.size setting is about how > much memory used as block cache used by StoreFile. I have no idea of > how HBase works internally. Typically, it is easy to understand that > increase the size of cache should help the read, but why it will harm > the write operation? The write performance down from 30,000 to 4,000 > for your reference, just by changing the hfile.block.cache.size from 0 to 0.4. > Could anyone give me a brief explanation about this observation or > give me some advices about what to study to understand what is block cache > used for? > > Another question: HBase write will first write to WAL then to memstore. > Will the write to WAL go to disk directly before hbase write memstore, > a sync operation or it is possible that write to WAL is still buffered > somewhere when hbase put the data into the memstore? > > Reading src code may cost me months, so a kindly reply will help me a > lot... ... > Thanks very much! > > Best Regards, > Ming >
