[
https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008477#comment-15008477
]
Walter Su commented on HDFS-8901:
---------------------------------
bq. (from description) this change uses ByteBuffer through the codes in
implementing striping position read. It will also fix avoiding unnecessary data
copying between striping read chunk buffers and decode input buffers.
I think, for PositionStripeReader, the data copy from buf\[\](given by caller)
to decodeInputs\[\]\[\] is unavoidable. An {{AlignedStripe}} actually merged
multiple real stripes. I mean if cellsize is 64kb, a real stripe is 6*64kb, an
{{AlignedStripe}} could be n*6*64kb (assume given buf\[\] is larger than that).
decodeInputs\[0\]\[n*64k\] is copied from n positions from buf\[\].
I understand native erasure coder prefers direct ByteBuffer. So we can make
decodeInputs\[\] a direct ByteBuffer array. Again we copy data from buf\[\] to
decodeInputs\[\] (off-heap bytebuffers).
So I think the changes to PositionStripeReader are good, and changes to
DFSInputStream are unnecessary.
> Use ByteBuffer in striping positional read
> ------------------------------------------
>
> Key: HDFS-8901
> URL: https://issues.apache.org/jira/browse/HDFS-8901
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HDFS-8901-v2.patch, initial-poc.patch
>
>
> Native erasure coder prefers to direct ByteBuffer for performance
> consideration. To prepare for it, this change uses ByteBuffer through the
> codes in implementing striping position read. It will also fix avoiding
> unnecessary data copying between striping read chunk buffers and decode input
> buffers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
