[
https://issues.apache.org/jira/browse/HDFS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029483#comment-14029483
]
Todd Lipcon commented on HDFS-3528:
-----------------------------------
Cool, thanks for starting on this. For the full improvement on the write path
we also will need a "bulk_calculate_crc32" equivalent to run on the client
side. Might make sense to split it up, though.
> Use native CRC32 in DFS write path
> ----------------------------------
>
> Key: HDFS-3528
> URL: https://issues.apache.org/jira/browse/HDFS-3528
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, hdfs-client, performance
> Affects Versions: 2.0.0-alpha
> Reporter: Todd Lipcon
> Assignee: James Thomas
> Attachments: HDFS-3528.patch
>
>
> HDFS-2080 improved the CPU efficiency of the read path by using native
> SSE-enabled code for CRC verification. Benchmarks of the write path show that
> it's often CPU bound by checksums as well, so we should make the same
> improvement there.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
