[
https://issues.apache.org/jira/browse/HBASE-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13818840#comment-13818840
]
Nicolas Liochon commented on HBASE-8323:
----------------------------------------
I looked at it further. It's possible to use DataChecksum. The issue is in
HBase code: the native implementation wants a DirectByteBuffer, but HBase uses
'standard' byte buffers... We need to change this if the want to use the native
implementation.
> Low hanging checksum improvements
> ---------------------------------
>
> Key: HBASE-8323
> URL: https://issues.apache.org/jira/browse/HBASE-8323
> Project: HBase
> Issue Type: Improvement
> Components: Performance
> Reporter: Enis Soztutar
>
> Over at Hadoop land, [~tlipcon] had done some improvements for checksums, a
> native implementation for CRC32C (HADOOP-7445) and bulk verify of checksums
> (HADOOP-7444).
> In HBase, we can do
> - Also develop a bulk verify API. Regardless of
> hbase.hstore.bytes.per.checksum we always want to verify of the whole
> checksum for the hfile block.
> - Enable NativeCrc32 to be used as a checksum algo. It is not clear how much
> gain we can expect over pure java CRC32.
> Though, longer term we should focus on convincing hdfs guys for inline
> checksums (HDFS-2699)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
