[ 
https://issues.apache.org/jira/browse/HBASE-27225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17569942#comment-17569942
 ] 

Wellington Chevreuil commented on HBASE-27225:
----------------------------------------------

{quote}
One interesting addition is the waistedBytes – I realized that if you choose 
block sizes that don't nicely divide the bucketCapacity you end up with left 
over space in each bucket which can add up to a lot in aggregate. One can use 
this information to try to choose bucket sizes which better divide into the 
bucketCapacity to reduce waste.
{quote}
I think this may be worsened by encoding/compression, when then block sizes can 
vary a lot according to the compression efficiency. This could then be better 
controlled by the "hbase.writer.unified.encoded.blocksize.ratio" property 
originally proposed on HBASE-17757.

> Add BucketAllocator bucket size statistic logging
> -------------------------------------------------
>
>                 Key: HBASE-27225
>                 URL: https://issues.apache.org/jira/browse/HBASE-27225
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Bryan Beaudreault
>            Assignee: Bryan Beaudreault
>            Priority: Major
>              Labels: patch-available
>
> BucketCache places blocks into configurably sized buckets based on the block 
> size. The default sizes aim to provide broad coverage for common block sizes, 
> but should probably be tuned for certain use-cases. However, we provide no 
> way for operators to gain insight into the distribution of buckets.
> There already exists a BucketAllocator#logStatistics method, but it is not 
> called anywhere. I suggest that we hook that up in BucketCache#logStats 
> (which is called periodically by a stats thread). We can go from there.
> Looking at the IndexStatistics used in that method, it looks like a good 
> start. One thing I'd like to add is a count of freeBuckets and 
> completelyFreeBuckets per index. I think this will be useful for indicating 
> how much more wiggle room we have for redistributing buckets among the 
> various block sizes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to