[
https://issues.apache.org/jira/browse/HBASE-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814737#comment-13814737
]
Nicolas Liochon commented on HBASE-8323:
----------------------------------------
I saw it while looking at a server. The class NativeCrc32 is not public in
hadoop common, (while PureJavaCrc32C is public, in the java & hadoop meanings
of the terms). The easiest way for HBase would be to reuse directly
NativeCrc32... [[email protected]], is there any problem if NativeCrc32 becomes
public, or at least LimitedPrivate?
> Low hanging checksum improvements
> ---------------------------------
>
> Key: HBASE-8323
> URL: https://issues.apache.org/jira/browse/HBASE-8323
> Project: HBase
> Issue Type: Improvement
> Components: Performance
> Reporter: Enis Soztutar
>
> Over at Hadoop land, [~tlipcon] had done some improvements for checksums, a
> native implementation for CRC32C (HADOOP-7445) and bulk verify of checksums
> (HADOOP-7444).
> In HBase, we can do
> - Also develop a bulk verify API. Regardless of
> hbase.hstore.bytes.per.checksum we always want to verify of the whole
> checksum for the hfile block.
> - Enable NativeCrc32 to be used as a checksum algo. It is not clear how much
> gain we can expect over pure java CRC32.
> Though, longer term we should focus on convincing hdfs guys for inline
> checksums (HDFS-2699)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
