[
https://issues.apache.org/jira/browse/HBASE-27232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17569634#comment-17569634
]
Bryan Beaudreault commented on HBASE-27232:
-------------------------------------------
Interesting, thanks for the explanation. That makes sense in terms of
estimating the amount of skew between these block sizes. Did you have any
indication in the BucketCache itself that changing this value helped it in some
way? Maybe more blocks fit in?
If you're experimenting with BucketCache you may also be interested in these 2
jiras I have open: https://issues.apache.org/jira/browse/HBASE-27229 and
https://issues.apache.org/jira/browse/HBASE-27225. The latter specifically
might relate to this block size skew.
> Fix checking for encoded block size when deciding if block should be closed
> ---------------------------------------------------------------------------
>
> Key: HBASE-27232
> URL: https://issues.apache.org/jira/browse/HBASE-27232
> Project: HBase
> Issue Type: Improvement
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> On HFileWriterImpl.checkBlockBoundary, we useed to consider the unencoded and
> uncompressed data size when deciding to close a block and start a new one.
> That could lead to varying "on-disk" block sizes, depending on the encoding
> efficiency for the cells in each block.
> HBASE-17757 introduced the hbase.writer.unified.encoded.blocksize.ratio
> property, as ration of the original configured block size, to be compared
> against the encoded size. This was an attempt to ensure homogeneous block
> sizes. However, the check introduced by HBASE-17757 also considers the
> unencoded size, which in the cases where encoding efficiency is higher than
> what's configured in hbase.writer.unified.encoded.blocksize.ratio, it would
> still lead to varying block sizes.
> This patch changes that logic, to only consider encoded size if
> hbase.writer.unified.encoded.blocksize.ratio property is set, otherwise, it
> will consider the unencoded size. This gives a finer control over the on-disk
> block sizes and the overall number of blocks when encoding is in use.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
