[
https://issues.apache.org/jira/browse/HBASE-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-10403:
---------------------------------
Attachment: HBASE-10403.0.patch
[~zjushch] you noticed :)
Here's a sketch of the direction I'm moving. A couple observation:
- the logic in CacheConfig#instantiateBlockCache is pretty convoluted, with
the different implementations all configured (or partially so) by the same if
block
- CombinedBlockCache and DoubleBlockCache are actually strategies for
implementing a 2-level cache. It would be nice, for instance, for a SlabCache
user to take advantage of the CombinedBlockCache strategy.
- A little cleanup here will let us experiment with other caching strategies.
IE, I'd like to try a 3-level cache with different implementations for index
blocks (L0), decompressed hot blocks (L1), and compressed cold/speculatively
read blocks (L2).
> Simplify offheap cache configuration
> ------------------------------------
>
> Key: HBASE-10403
> URL: https://issues.apache.org/jira/browse/HBASE-10403
> Project: HBase
> Issue Type: Bug
> Components: io
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Priority: Minor
> Attachments: HBASE-10403.0.patch
>
>
> The BucketCache (HBASE-7404) is a very nice piece of functionality which is
> hidden behind complex configuration. Enabling it currently requires manual
> calculation of L1 cache. It'd be nice to make this easier to use and conform
> better with the existing heap management tools we already have.
> Turning it on currently requires explicitly setting LruBlockCache (L1)
> instance size and IOEngine (L2) size, making sure that L1 size isn't too big
> vs global memstore and total heap. This is further confused by
> hbase.bucketcache.size accepting a percentage of total heap OR explicit size
> in MB. Enabling SlabCache is slightly easier in that it just accepts whatever
> LruBlockCache is provided.
> Turning on BucketCache using off-heap mode could look like:
> hbase-env.sh:
> Set HBASE_REGIONSERVER_OPTS:
> -Xmx5000m
> -XX:MaxDirectMemorySize=15000m
> hbase-site.xml:
> - hbase.regionserver.global.memstore.size = 0.7
> - hbase.regionserver.onheap.blockcache.size = 0.1
> - hbase.regionserver.blockcache.impl = BucketCache
> - hbase.bucketcache.ioengine = offheap
> The result being a CombinedCache instance with 500m LruBlockCache + 15000m
> ByteBufferIOEngine running in direct mode.
> This example does a couple things (mostly for the admin):
> - knows NOT to enable SlabCache
> - s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
> - maintains the validity of HBaseConfiguration's existing check that global
> MemStore + LruBlockCache == 0.8
> - maps "BucketCache" into meaning "a CombinedCache instance with these
> implementations for L1 and L2."
> - Figures out appropriate values for hbase.bucketcache.size and
> hbase.bucketcache.percentage.in.combinedcache
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
