HBase block cache does not expose any statistics on a per-table basis. This looks like a nice to have feature in a next releases. If your load is mostly random reads, than several factors affect the overall performance:
1. The avg read size / block size ratio. The large ratio - the better block cache hit ration you will have. 2. Hot data set size. 3 Available RAM for block cache For example, Block size = 64K Avg read size = 1K Hot data set size = 1GB You will need 64GB of block cache to cache 100% your random reads. Usually, for random reads the smaller block size - the better, but you can not decrease block size w/o additional overhead of bookkeeping larger number of smaller blocks. Your actual limit is around 16KB. Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: [email protected] ________________________________________ From: Vaibhav Puranik [[email protected]] Sent: Friday, October 18, 2013 3:20 PM To: [email protected] Subject: Block cache hit ratio question Hi all, Our HBase is slowing down because of lots of disk reads. We are trying to analyze it. We have lots of real time random reads. And We found that HBase is constantly reading from disk instead of from memory. Looks like block cache hit ratio is a good place to start. Is there any way we can get block cache hit ratio by table or by region? I could only find it per region server in region server UI. I couldn't find it in API either. Feel free to suggest any other metrics we should be looking at too. Thanks, Vaibhav GumGum Confidentiality Notice: The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or [email protected] and delete or destroy any copy of this message and its attachments.
