[
https://issues.apache.org/jira/browse/HDFS-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113917#comment-14113917
]
Todd Lipcon commented on HDFS-3529:
-----------------------------------
Hey Yi. Please note that [~james.thomas] has been doing work recently so that
native CRC32 can run against byte[] arrays instead of just against byte
buffers. So, the major performance improvement enabled by this JIRA probably
isn't as relevant anymore.
Using direct buffers might save us a memcpy, but for small cache-resident
buffer sizes, my guess is that we won't see a big boost. I'd be interested to
see some results, though, if you have a prototype of the patch against trunk.
> Use direct buffers for data in write path
> -----------------------------------------
>
> Key: HDFS-3529
> URL: https://issues.apache.org/jira/browse/HDFS-3529
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, performance
> Affects Versions: 2.0.0-alpha
> Reporter: Todd Lipcon
> Assignee: Trevor Robinson
> Attachments: HDFS-3529.patch, dfsio-x86-trunk-vs-3529.png
>
>
> The write path currently makes several unnecessary data copies in order to go
> to and from byte arrays. We can improve performance by using direct byte
> buffers to avoid the copy. This is also a prerequisite for native checksum
> calculation (HDFS-3528)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
