[
https://issues.apache.org/jira/browse/HBASE-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13818931#comment-13818931
]
Jean-Marc Spaggiari commented on HBASE-8323:
--------------------------------------------
Will a buffer conversion benefit from the CRC improvement? I mean, if we keept
the 'standard' byte buffers, but convert them to DirectByteBuffer when we call
DataCheckSum, will the cost of this conversion be lower than the benefit of the
DataCheckSum? Is so, that might be an easy change. Else, if we have to chance
to DirectByteBuffer all over the place, might be a bit more difficult...
> Low hanging checksum improvements
> ---------------------------------
>
> Key: HBASE-8323
> URL: https://issues.apache.org/jira/browse/HBASE-8323
> Project: HBase
> Issue Type: Improvement
> Components: Performance
> Reporter: Enis Soztutar
>
> Over at Hadoop land, [~tlipcon] had done some improvements for checksums, a
> native implementation for CRC32C (HADOOP-7445) and bulk verify of checksums
> (HADOOP-7444).
> In HBase, we can do
> - Also develop a bulk verify API. Regardless of
> hbase.hstore.bytes.per.checksum we always want to verify of the whole
> checksum for the hfile block.
> - Enable NativeCrc32 to be used as a checksum algo. It is not clear how much
> gain we can expect over pure java CRC32.
> Though, longer term we should focus on convincing hdfs guys for inline
> checksums (HDFS-2699)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
