[
https://issues.apache.org/jira/browse/HADOOP-11938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537768#comment-14537768
]
Kai Zheng commented on HADOOP-11938:
------------------------------------
Hi [~hitliuyi],
Thanks for your review and the great comments. They're really helpful to make
the solid base. My following patch will address most of them. Thanks!
bq. What's the reason we need to reset inputbuffer? Input buffers are given by
caller.
Yes for now we don't need it. I got it from HADOOP-11847, there it's needed to
reset input buffers provided from reused internal buffers. I will remove it and
only add it when really necessary then.
bq. all the input have the same length besides some inputs are null
For now all the inputs shall not be null. In HADOOP-11847 some inputs can be
null to indicate not to read those units. I would really add such checks in
HADOOP-11847 because it's not trivial to do such checks. This patch would be
better to focus on the ByteBuffer supporting.
bq. We should also enhance the description in RawErasureDecoder#decode
Yes it should . I did it in HADOOP-11847 already. Could we leave it now in this
issue?
Thanks a lot!
> Fix ByteBuffer version encode/decode API of raw erasure coder
> -------------------------------------------------------------
>
> Key: HADOOP-11938
> URL: https://issues.apache.org/jira/browse/HADOOP-11938
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-11938-HDFS-7285-v1.patch,
> HADOOP-11938-HDFS-7285-workaround.patch
>
>
> While investigating a test failure in {{TestRecoverStripedFile}}, one issue
> in raw erasrue coder, caused by an optimization in below codes. It assumes
> the heap buffer backed by the bytes array available for reading or writing
> always starts with zero and takes the whole space.
> {code}
> protected static byte[][] toArrays(ByteBuffer[] buffers) {
> byte[][] bytesArr = new byte[buffers.length][];
> ByteBuffer buffer;
> for (int i = 0; i < buffers.length; i++) {
> buffer = buffers[i];
> if (buffer == null) {
> bytesArr[i] = null;
> continue;
> }
> if (buffer.hasArray()) {
> bytesArr[i] = buffer.array();
> } else {
> throw new IllegalArgumentException("Invalid ByteBuffer passed, " +
> "expecting heap buffer");
> }
> }
> return bytesArr;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
