[
https://issues.apache.org/jira/browse/HBASE-17204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124992#comment-16124992
]
Anoop Sam John commented on HBASE-17204:
----------------------------------------
[~stack]
I was analyzing ur suggestion on config options for the off heap sizes for read
and write paths. Whether this should be some % of total max off heap size for
the process. As u know, as of now both parts have configs using which one can
specify the exact size to be used. (Not in %)
Write side - hbase.regionserver.offheap.global.memstore.size - Size in MBs
Read side - hbase.bucketcache.size To be specified in MBs size. This is like
this from day 1 (Before 11425). The config allows to specify size as % but that
is % of the heap size (Xmx) means it can be used with on heap BC only. For file
mode and off heap mode it has to be in exact MBs size (There is no sanity check
for this in code. Also we need and better documentation)
As u said we can change these to allow % of Max Direct memory size. But will
that be too complicated for a user is what my thinking now. Lets see all Off
heap memory using parts in RS
1. Off heap BC size
2. Off heap MSLAB size
3. Off heap RPC side buffer pool
4. Misc - Netty needs it. Java NIO needs it(RPC side as well as file read
side).
When one config the BC size and MSLAB size as % of Max Direct memory, they have
to have a better judging what extra memory is needed for this misc and buffer
pool. The Buffer pool need is fixed as that will have a fixed number of
constant sized buffers (def to 64 KB and 2x handler count BBs). The misc need
is also a headache. Instead they will be knowing the size needs of each of
these BC and MSLAB.(How many GBs for each)
We already allowed the off heap BC size to specified as exact MBs size. Seems
many users using it already. Will be a BC break and better we can avoid if
possible.
So IMO we can continue with current way of exact size specify using the above 2
configs
> Make L2 off heap cache default ON
> ---------------------------------
>
> Key: HBASE-17204
> URL: https://issues.apache.org/jira/browse/HBASE-17204
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Blocker
> Fix For: 2.0.0
>
>
> L2 cache can be used for data blocks. By default it is off now. After
> HBASE-11425 work, L2 off heap cache can equally perform with L1 on heap
> cache. On heavy loaded workload, this can even out perform L1 cache. Pls see
> recently published report by Alibaba. Also this work was backported by
> Rocketfuel and similar perf improvement report from them too.
> Let us turn L2 off heap cache ON. As it is off heap, we can have much larger
> sized L2 BC. What should be the default size?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
