[
https://issues.apache.org/jira/browse/HBASE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541755#comment-14541755
]
Anoop Sam John commented on HBASE-11927:
----------------------------------------
Patch looks good.
bq.DataChecksum.Type.valueOf(cktype.getCode())
Do we need a mapping function which maps HBase ChecksumType to hadoop
DataChecksum ? I agree that both the type codes are same so no issue now.
Still that may be cleaner IMO.
We had a fallback mechanism to Java checksum when the hadoop library is not
available. So this we are removing now. Make it clear in Release Notes (about
the expectation)
Good that we no longer have expectation of Block data been a byte[] [This one
- checksumObject.update(byte[],int,int) ]... Will help in offheap read path.
:-)
> Use Native Hadoop Library for HFile checksum (And flip default from CRC32 to
> CRC32C)
> ------------------------------------------------------------------------------------
>
> Key: HBASE-11927
> URL: https://issues.apache.org/jira/browse/HBASE-11927
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Apekshit Sharma
> Attachments: HBASE-11927-v1.patch, HBASE-11927-v2.patch,
> HBASE-11927-v4.patch, HBASE-11927.patch, after-compact-2%.svg,
> after-randomWrite1M-0.5%.svg, before-compact-22%.svg,
> before-randomWrite1M-5%.svg, c2021.crc2.svg, c2021.write.2.svg,
> c2021.zip.svg, crc32ct.svg
>
>
> Up in hadoop they have this change. Let me publish some graphs to show that
> it makes a difference (CRC is a massive amount of our CPU usage in my
> profiling of an upload because of compacting, flushing, etc.). We should
> also make use of native CRCings -- especially the 2.6 HDFS-6865 and ilk -- in
> hbase but that is another issue for now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
