[
https://issues.apache.org/jira/browse/HDFS-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485869#comment-13485869
]
Trevor Robinson commented on HDFS-3529:
---------------------------------------
The two Jenkins failures seem to be the same issue:
# One {{BlockReceiver$PacketResponder}} thread is interrupted in flushing an
ack to an upstream datanode.
# Another {{BlockReceiver$PacketResponder}} thread gets an {{EOFException}}
trying to read the length prefix of an ack.
# An NPE occurs during {{PacketReceiver.mirrorPacketTo}} (called via
{{DataXceiver.writeBlock}}) because the packet buffer is null, which should
only happen if {{close}} has been called.
I'm not sure which of the three caused the others, or if there's some other
cause. I ran hundreds of iterations of {{TestLoadGenerator}} in a loop, and
cannot reproduce a failure locally. I'd appreciate any suggestions on how to go
about reproducing it. For example, is it reasonable to attach a patch with
extra logging, or does that disrupt builds for other issues? Or is it possible
to get access to a machine with the same configuration?
> Use direct buffers for data in write path
> -----------------------------------------
>
> Key: HDFS-3529
> URL: https://issues.apache.org/jira/browse/HDFS-3529
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node, performance
> Affects Versions: 2.0.0-alpha
> Reporter: Todd Lipcon
> Assignee: Trevor Robinson
> Attachments: HDFS-3529.patch
>
>
> The write path currently makes several unnecessary data copies in order to go
> to and from byte arrays. We can improve performance by using direct byte
> buffers to avoid the copy. This is also a prerequisite for native checksum
> calculation (HDFS-3528)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
