On Mon, Jun 1, 2015 at 2:24 AM, Dejan Menges <[email protected]> wrote:
> Hi, > > What's the best way to monitor / know how's bucket cache being used, how > much stuff is cached there, etc? > > See the UI on a regionserver. Look down the page to the 'Block Cache' section. It has detail on both onheap and LRU offheap. See also the documentation on offheap cache: http://hbase.apache.org/book.html#offheap.blockcache Look also at metrics where we report block cache stats as well as offheap used by the JVM. > Our RegionServer can use 32G of heap size, so we exported HBASE_OFFHEAPSIZE > to 24G in hbase-env.sh, set hfile.block.cache.size to 0.05, and set couple > of block sizes that we know we are using knowing our usage patterns. And > this is where strange part starts - in web UI we see now, with turning this > off, Turning off what? > with those values, that total BlockCache available is 1G - before it > was 10G. What we basically tried to achieve was to double it to 20G. > > Are you sure this not the onheap portion of BlockCache? > Documentation we were referring to was > > http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/admin_hbase_blockcache_configure.html#concept_cp3_fhy_dr_unique_1__section_m3r_2cz_dr_unique_1 > as HBase book is not going into too much details how to properly configure > this and get what you want. > > Please let us know what is missing from here: http://hbase.apache.org/book.html#offheap.blockcache We'd like to fix it. > Btw. if we put hfile.block.cache.size to 0.2 we see in web UI that total > available cache is 24G, but then after some time we had region server > crashing. Host server have enough RAM, considering only data node and > region server running there (128G of RAM in total) so we thought we could > increase caching with turning on this functionality. > > Make sure your hbase has HBASE-11678 > Do you maybe see what exactly we are doing wrong? How to exactly increase > offheap caching - and is it possible to monitor it anyhow, as in metrics we > don't see anything associating to it? > > Please provide more detail. Your configurations and what you've changed in hbase-env.sh. St.Ack > Thanks, > Dejan >
