[
https://issues.apache.org/jira/browse/HADOOP-11938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539276#comment-14539276
]
Kai Zheng commented on HADOOP-11938:
------------------------------------
Thanks Yi for the complete detailed review!
bq. dataLen/iPos/oPos are not necessary. we can use buffer.limit(), position()
instead.
I agree.
bq. I wonder whether this works, I see it only decode for output\[0\].
Yes. XOR coder can only recover one erasure of unit. There are tests for
recovering one unit. Please note for this limit XOR coder will not
independently used, instead can be utilized by other coders, like HH coder.
bq. besides idx1/idx2, others are unnecessary, then the code is more clear.
One reason I named the variable is to reduce the lengthy code line that
otherwise has to spread to multiple lines not naturally at all. Using the
variables would make the relevant statements/expressions very compact, also
avoiding too many nesting. Does this make sense? I'm OK not to use them if you
insist. Thanks.
bq. The length of inputs/outputs is not equal to chunksize, we can still decode.
Good idea! Per our latest discussion we need to support variable width of data
as inputs, we need to add such tests.
bq. Some negative test, we can catch the expected exception.
Yes I can do that.
> Fix ByteBuffer version encode/decode API of raw erasure coder
> -------------------------------------------------------------
>
> Key: HADOOP-11938
> URL: https://issues.apache.org/jira/browse/HADOOP-11938
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-11938-HDFS-7285-v1.patch,
> HADOOP-11938-HDFS-7285-workaround.patch
>
>
> While investigating a test failure in {{TestRecoverStripedFile}}, one issue
> in raw erasrue coder, caused by an optimization in below codes. It assumes
> the heap buffer backed by the bytes array available for reading or writing
> always starts with zero and takes the whole space.
> {code}
> protected static byte[][] toArrays(ByteBuffer[] buffers) {
> byte[][] bytesArr = new byte[buffers.length][];
> ByteBuffer buffer;
> for (int i = 0; i < buffers.length; i++) {
> buffer = buffers[i];
> if (buffer == null) {
> bytesArr[i] = null;
> continue;
> }
> if (buffer.hasArray()) {
> bytesArr[i] = buffer.array();
> } else {
> throw new IllegalArgumentException("Invalid ByteBuffer passed, " +
> "expecting heap buffer");
> }
> }
> return bytesArr;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
