Nick Dimiduk created HBASE-10403:
------------------------------------
Summary: Simplify offheap cache configuration
Key: HBASE-10403
URL: https://issues.apache.org/jira/browse/HBASE-10403
Project: HBase
Issue Type: Bug
Components: io
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
The BucketCache (HBASE-7404) is a very nice piece of functionality which is
hidden behind complex configuration. Enabling it currently requires manual
calculation of L1 cache. It'd be nice to make this easier to use and conform
better with the existing heap management tools we already have.
Turning it on currently requires explicitly setting LruBlockCache (L1) instance
size and IOEngine (L2) size, making sure that L1 size isn't too big vs global
memstore and total heap. This is further confused by hbase.bucketcache.size
accepting a percentage of total heap OR explicit size in MB. Enabling SlabCache
is slightly easier in that it just accepts whatever LruBlockCache is provided.
Turning on BucketCache using off-heap mode could look like:
hbase-env.sh:
Set HBASE_REGIONSERVER_OPTS:
-Xmx5000m
-XX:MaxDirectMemorySize=15000m
hbase-site.xml:
- hbase.regionserver.global.memstore.size = 0.7
- hbase.regionserver.onheap.blockcache.size = 0.1
- hbase.regionserver.blockcache.impl = BucketCache
- hbase.bucketcache.ioengine = offheap
The result being a CombinedCache instance with 500m LruBlockCache + 15000m
ByteBufferIOEngine running in direct mode.
This example does a couple things (mostly for the admin):
- knows NOT to enable SlabCache
- s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
- maintains the validity of HBaseConfiguration's existing check that global
MemStore + LruBlockCache == 0.8
- maps "BucketCache" into meaning "a CombinedCache instance with these
implementations for L1 and L2."
- Figures out appropriate values for hbase.bucketcache.size and
hbase.bucketcache.percentage.in.combinedcache
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
