Also note that configuration is slightly changed between 0.98 and 1.0, see HBASE-11520. From the release note:
> Remove "hbase.bucketcache.percentage.in.combinedcache". Simplifies config of block cache. If you are using this config., after > this patch goes in, it will be ignored. The L1 LruBlockCache will be whatever hfile.block.cache.size is set to and the L2 > BucketCache will be whatever hbase.bucketcache.size is set to. On Mon, Jun 1, 2015 at 8:10 AM, Stack <[email protected]> wrote: > On Mon, Jun 1, 2015 at 2:24 AM, Dejan Menges <[email protected]> > wrote: > > > Hi, > > > > What's the best way to monitor / know how's bucket cache being used, how > > much stuff is cached there, etc? > > > > > See the UI on a regionserver. Look down the page to the 'Block Cache' > section. It has detail on both onheap and LRU offheap. See also the > documentation on offheap cache: > http://hbase.apache.org/book.html#offheap.blockcache Look also at metrics > where we report block cache stats as well as offheap used by the JVM. > > > > > Our RegionServer can use 32G of heap size, so we exported > HBASE_OFFHEAPSIZE > > to 24G in hbase-env.sh, set hfile.block.cache.size to 0.05, and set > couple > > of block sizes that we know we are using knowing our usage patterns. And > > this is where strange part starts - in web UI we see now, with turning > this > > off, > > > Turning off what? > > > > > with those values, that total BlockCache available is 1G - before it > > was 10G. What we basically tried to achieve was to double it to 20G. > > > > > Are you sure this not the onheap portion of BlockCache? > > > > > Documentation we were referring to was > > > > > http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/admin_hbase_blockcache_configure.html#concept_cp3_fhy_dr_unique_1__section_m3r_2cz_dr_unique_1 > > as HBase book is not going into too much details how to properly > configure > > this and get what you want. > > > > > Please let us know what is missing from here: > http://hbase.apache.org/book.html#offheap.blockcache We'd like to fix it. > > > > > Btw. if we put hfile.block.cache.size to 0.2 we see in web UI that total > > available cache is 24G, but then after some time we had region server > > crashing. Host server have enough RAM, considering only data node and > > region server running there (128G of RAM in total) so we thought we could > > increase caching with turning on this functionality. > > > > > Make sure your hbase has HBASE-11678 > > > > > Do you maybe see what exactly we are doing wrong? How to exactly increase > > offheap caching - and is it possible to monitor it anyhow, as in metrics > we > > don't see anything associating to it? > > > > > Please provide more detail. Your configurations and what you've changed in > hbase-env.sh. > > St.Ack > > > > > Thanks, > > Dejan > > >
