[
https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-17757:
-------------------------------
Release Note: Blocksize is set in columnfamily's atrributes. It is used to
control block sizes when generating blocks. But, it doesn't take encoding into
count. If you set encoding to blocks, after encoding, the block size varies.
Since blocks will be cached in memory after encoding (default), it will cause
memory fragment if using blockcache, or decrease the pool efficiency if using
bucketCache. This issue introduced a new config named
'hbase.writer.unified.encoded.blocksize.ratio'. The default value of this
config is 1, meaning doing nothing. If this value is set to a smaller value
like 0.5, and the blocksize is set to 64KB(default value of blocksize). It will
unify the blocksize after encoding to 64KB * 0.5 = 32KB. Unified blocksize will
releaf the memory problems mentioned above.
> Unify blocksize after encoding to decrease memory fragment
> -----------------------------------------------------------
>
> Key: HBASE-17757
> URL: https://issues.apache.org/jira/browse/HBASE-17757
> Project: HBase
> Issue Type: New Feature
> Reporter: Allan Yang
> Assignee: Allan Yang
> Attachments: HBASE-17757.patch, HBASE-17757v2.patch
>
>
> Usually, we store encoded block(uncompressed) in blockcache/bucketCache.
> Though we have set the blocksize, after encoding, blocksize is varied. Varied
> blocksize will cause memory fragment problem, which will result in more FGC
> finally.In order to relief the memory fragment, This issue adjusts the
> encoded block to a unified size.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
