[
https://issues.apache.org/jira/browse/HBASE-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819265#comment-13819265
]
Enis Soztutar commented on HBASE-8323:
--------------------------------------
bq. The issue is in HBase code: the native implementation wants a
DirectByteBuffer, but HBase uses 'standard' byte buffers
Combining this with off-heap cache might make sense if we can read the block to
an off-heap buffer, we can do the bulk verify at the same time.
> Low hanging checksum improvements
> ---------------------------------
>
> Key: HBASE-8323
> URL: https://issues.apache.org/jira/browse/HBASE-8323
> Project: HBase
> Issue Type: Improvement
> Components: Performance
> Reporter: Enis Soztutar
>
> Over at Hadoop land, [~tlipcon] had done some improvements for checksums, a
> native implementation for CRC32C (HADOOP-7445) and bulk verify of checksums
> (HADOOP-7444).
> In HBase, we can do
> - Also develop a bulk verify API. Regardless of
> hbase.hstore.bytes.per.checksum we always want to verify of the whole
> checksum for the hfile block.
> - Enable NativeCrc32 to be used as a checksum algo. It is not clear how much
> gain we can expect over pure java CRC32.
> Though, longer term we should focus on convincing hdfs guys for inline
> checksums (HDFS-2699)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
