[
https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902914#comment-15902914
]
Allan Yang edited comment on HBASE-17757 at 3/9/17 11:31 AM:
-------------------------------------------------------------
{quote}
Why would we not just enable this behavior always? What feedback do i get that
tells me I've chosen a bad UNIFY_ENCODED_BLOCKSIZE_RATIO ? Thank you.
{quote}
For long time, blockszie is stand for block size before encoding, user may have
carefully chose a blocksize to balance the performance. if we enable this
feature always, after upgrading their HBase to a new version with this patch,
the actual blocksize will be changed. It may not be friendly to some users. So
in our company, we made a switch, only deploying this feature for new
customers. Old customers who want to use this feature need to carefully choose
a UNIFY_ENCODED_BLOCKSIZE_RATIO .
{quote}
What feedback do i get that tells me I've chosen a bad
UNIFY_ENCODED_BLOCKSIZE_RATIO ?
{quote}
No feedback, I'm afraid. Like blocksize, HBase users is responsible for their
choose. A bad 'blocksize' or ' UNIFY_ENCODED_BLOCKSIZE_RATIO' may compromise
performance, but no other damage.
{quote}
Can we just reduce the #configs here? Say by default we want this feature to
be off then give this ratio as 1 only?
{quote}
[~anoopsamjohn], some problem here, blockszie is stand for block size before
encoding, if we use only ratio to control this feature, blocksize will become
block size after encoding
After all, If we add a new config for hfile block, for example,
'encodedBlocksize', if it is set, it will override 'blocksize', then we can
introduce only one new config. What do you think? [~stack], [~anoopsamjohn]
was (Author: allan163):
{quote}
Why would we not just enable this behavior always? What feedback do i get that
tells me I've chosen a bad UNIFY_ENCODED_BLOCKSIZE_RATIO ? Thank you.
{quote}
For long time, blockszie is stand for block size before encoding, user may have
carefully chose a blocksize to balance the performance. if we enable this
feature always, after upgrading their HBase to a new version with this patch,
the actual blocksize will be changed. It may not be friendly to some users. So
in our company, we made a switch, only deploying this feature for new
customers. Old customers who want to use this feature need to carefully choose
a UNIFY_ENCODED_BLOCKSIZE_RATIO .
{quote}
What feedback do i get that tells me I've chosen a bad
UNIFY_ENCODED_BLOCKSIZE_RATIO ?
{quote}
No feedback, I'm afraid. Like blocksize, HBase users is responsible for their
choose. A bad 'blocksize' or ' UNIFY_ENCODED_BLOCKSIZE_RATIO' may compromise
performance, but no other damage.
{quote}
Can we just reduce the #configs here? Say by default we want this feature to
be off then give this ratio as 1 only?
{quote}
[~anoopsamjohn], some problem here, blockszie is stand for block size before
encoding, if we use only ratio to control this feature, blocksize will become
block size after encoding
After all, If we add a new config for hfile block, for example,
'encodedBlocksize', if it is set, it will override 'blocksize', then we can
introduce only one new config.
> Unify blocksize after encoding to decrease memory fragment
> -----------------------------------------------------------
>
> Key: HBASE-17757
> URL: https://issues.apache.org/jira/browse/HBASE-17757
> Project: HBase
> Issue Type: New Feature
> Reporter: Allan Yang
> Assignee: Allan Yang
> Attachments: HBASE-17757.patch
>
>
> Usually, we store encoded block(uncompressed) in blockcache/bucketCache.
> Though we have set the blocksize, after encoding, blocksize is varied. Varied
> blocksize will cause memory fragment problem, which will result in more FGC
> finally.In order to relief the memory fragment, This issue adjusts the
> encoded block to a unified size.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
