[
https://issues.apache.org/jira/browse/HBASE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040519#comment-13040519
]
Karthick Sankarachary commented on HBASE-3927:
----------------------------------------------
More often than not, the uncompressed bytes should be equal to the
"hfile.min.blocksize.size" setting, if I understand it correctly. Typically,
the {{HFile#Writer}} will close a block if its {{checkBlockBoundary}} method
throws an exception, which happens when the current block's size goes over. I
believe the only hfile block that can potentially have fewer (uncompressed)
bytes is the last one (which was current at the time of close). If so, it would
be nicer to expose the compression ratio (along with the total compressed
bytes) in the web UI.
> display total uncompressed byte size of a region in web UI
> ----------------------------------------------------------
>
> Key: HBASE-3927
> URL: https://issues.apache.org/jira/browse/HBASE-3927
> Project: HBase
> Issue Type: Improvement
> Components: metrics
> Reporter: Matt Corgan
> Assignee: Ted Yu
> Priority: Minor
> Attachments: 3927.txt
>
>
> The decision to split data blocks when flushing and compacting is made based
> on the uncompressed data size which can often lead to compressed disk blocks
> that are a fraction of the intended 64 KB (default). This often leads to a
> larger number of blocks and index entries than expected and can cause block
> indexes to take up GB of memory.
> There is already a "long totalUncompressedBytes" written to the HFileTrailer.
> It would be nice to expose this in the web UI to make it easier to calculate
> the compression ratio and then raise the block size appropriately (not
> necessarily to get it back to 64K).
> This should probably be added wherever the other HFile metrics are:
> RegionLoad.createRegions(..), and HServerLoad. HServerLoad is a Writable, so
> it may break serialization.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
