[
https://issues.apache.org/jira/browse/HBASE-27049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538966#comment-17538966
]
Andrew Kyle Purtell commented on HBASE-27049:
---------------------------------------------
The development of an HFile v4 could be beneficial for a number of reasons:
- It would let us improve the hfile block header structure. It should be
converted into protobuf encoded structures. Or keep the header a simple fixed
structure and add a footer that is protobuf encoded. Then in the future we
could make some types of changes without breaking readers. Developer effort
would be lower too, a change would involve updating the proto file and a few
lines of code. Today the same type of change would require updating many lines
of code.
- It would open a window for improving efficiency of compression and
encryption. If the binary output must change as a result, fine, it's a new file
format already. I realize it is a larger scope that what you are thinking now
[~chenfengge] but your concerns fit in here.
- It would let us introduce to the HFile some modern features of database data
files like counts and statistics, as a new type of meta block.
- Currently HFiles have indexes optimized for binary search but could
consider/support alternate indexes like compressed bitmaps.
> Decrease memory copy when decompress data
> -----------------------------------------
>
> Key: HBASE-27049
> URL: https://issues.apache.org/jira/browse/HBASE-27049
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: chenfengge
> Priority: Minor
>
> HBase RegionServer use createDecompressionStream in class
> org.apache.hadoop.hbase.io.compress.Compression, which cause extra memory
> copy during decompression.We can offer interface for block decompression,
> like "void decompress(ByteBuff src, ByteBuff dst);", and offer default
> implementation for all algorithms.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
