[
https://issues.apache.org/jira/browse/HADOOP-10778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065902#comment-14065902
]
Tsz Wo Nicholas Sze commented on HADOOP-10778:
----------------------------------------------
> Using crcutil will get us to 1 cycle per byte. But, my implementation I
> linked to above actually improves us to 0.73 cycles/byte for the chunked
> case, because we can compute the checksums in parallel (on different ALUs)
> from each chunk and don't need any kind of "combine" operation at the end.
Hey, the benchmark in mac does show that the zip (possibly crcutil) is much
fast than our native implementation. So, I guess there are still much room for
improvement in our native implementation.
Anyway, I take you point that zip may not be faster in some platform. Let me
update that patch.
> Use NativeCrc32 only if it is faster
> ------------------------------------
>
> Key: HADOOP-10778
> URL: https://issues.apache.org/jira/browse/HADOOP-10778
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Attachments: c10778_20140702.patch
>
>
> From the benchmark post in [this
> comment|https://issues.apache.org/jira/browse/HDFS-6560?focusedCommentId=14044060&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14044060],
> NativeCrc32 is slower than java.util.zip.CRC32 for Java 7 and above when
> bytesPerChecksum > 512.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
