[
https://issues.apache.org/jira/browse/HBASE-27225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17569942#comment-17569942
]
Wellington Chevreuil commented on HBASE-27225:
----------------------------------------------
{quote}
One interesting addition is the waistedBytes – I realized that if you choose
block sizes that don't nicely divide the bucketCapacity you end up with left
over space in each bucket which can add up to a lot in aggregate. One can use
this information to try to choose bucket sizes which better divide into the
bucketCapacity to reduce waste.
{quote}
I think this may be worsened by encoding/compression, when then block sizes can
vary a lot according to the compression efficiency. This could then be better
controlled by the "hbase.writer.unified.encoded.blocksize.ratio" property
originally proposed on HBASE-17757.
> Add BucketAllocator bucket size statistic logging
> -------------------------------------------------
>
> Key: HBASE-27225
> URL: https://issues.apache.org/jira/browse/HBASE-27225
> Project: HBase
> Issue Type: Improvement
> Reporter: Bryan Beaudreault
> Assignee: Bryan Beaudreault
> Priority: Major
> Labels: patch-available
>
> BucketCache places blocks into configurably sized buckets based on the block
> size. The default sizes aim to provide broad coverage for common block sizes,
> but should probably be tuned for certain use-cases. However, we provide no
> way for operators to gain insight into the distribution of buckets.
> There already exists a BucketAllocator#logStatistics method, but it is not
> called anywhere. I suggest that we hook that up in BucketCache#logStats
> (which is called periodically by a stats thread). We can go from there.
> Looking at the IndexStatistics used in that method, it looks like a good
> start. One thing I'd like to add is a count of freeBuckets and
> completelyFreeBuckets per index. I think this will be useful for indicating
> how much more wiggle room we have for redistributing buckets among the
> various block sizes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)