[
https://issues.apache.org/jira/browse/HADOOP-11938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534560#comment-14534560
]
Kai Zheng commented on HADOOP-11938:
------------------------------------
I had an off-line discussion with [~hitliuyi] and was convinced that we'd
better support arbitrary ByteBuffer (with any position) for the encode/decode
input/output buffers, without causing performance loss by using two parameters
{{offset}} and {{len}} when converting on-heap ByteBuffer to an bytes array:
{code}
bytesArr = bytebuffer.array();
offset = bytebuffer.position();
len = bytebuffer.remaining();
encode(bytebuffer1) =>
doEncode(bytesArr, offset, len)
{code}
As the change isn't trivial and takes time, it would be good to commit the work
around patch first in order to fix the failure sooner, and leave this issue
open for the final solution.
Thanks Yi for the great discussion!
> Fix ByteBuffer version encode/decode API of raw erasure coder
> -------------------------------------------------------------
>
> Key: HADOOP-11938
> URL: https://issues.apache.org/jira/browse/HADOOP-11938
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-11938-HDFS-7285-workaround.patch
>
>
> While investigating a test failure in {{TestRecoverStripedFile}}, one issue
> in raw erasrue coder, caused by an optimization in below codes. It assumes
> the heap buffer backed by the bytes array available for reading or writing
> always starts with zero and takes the whole space.
> {code}
> protected static byte[][] toArrays(ByteBuffer[] buffers) {
> byte[][] bytesArr = new byte[buffers.length][];
> ByteBuffer buffer;
> for (int i = 0; i < buffers.length; i++) {
> buffer = buffers[i];
> if (buffer == null) {
> bytesArr[i] = null;
> continue;
> }
> if (buffer.hasArray()) {
> bytesArr[i] = buffer.array();
> } else {
> throw new IllegalArgumentException("Invalid ByteBuffer passed, " +
> "expecting heap buffer");
> }
> }
> return bytesArr;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
