[
https://issues.apache.org/jira/browse/TEZ-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126989#comment-17126989
]
Jonathan Turner Eagles commented on TEZ-1719:
---------------------------------------------
Just know to be careful when enabling this as modern disks write in the
background and will return write success. In my case data came was read as all
zeros from intermediate spill
> Allow IFile reducer merge-sort to disable crc32 checksums
> ---------------------------------------------------------
>
> Key: TEZ-1719
> URL: https://issues.apache.org/jira/browse/TEZ-1719
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Gopal Vijayaraghavan
> Assignee: László Bodor
> Priority: Major
> Labels: Performance
>
> Next-gen filesystems like BTRFS and ZFS provide their own checksumming for
> disk data.
> Using PureJavaCrc32 for data written for temporary spills to such filesystems
> is a complete waste of CPU resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
