[
https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520899#comment-14520899
]
Zhe Zhang commented on HDFS-7348:
---------------------------------
Thanks Yi for the explanation.
bq. Current implementation the recovery node is one of the source node
Yes that sounds reasonable. I initially thought the recovery node should be the
target, since writing locally saves more bandwidth than reading locally --
writing locally avoid transferring a whole block while reading locally avoids
transferring 1/6 of a block. But in some cases the user might want to choose
reading locally.
This patch reads and writes remotely. I think it's a generic solution and a
good starting point. We can do local writing and local reading logics as
follow-on under HDFS-8031.
HADOOP-11847 actually has a very large patch and it might take longer than this
patch to review and commit. I think it's much easier to prepare a full input
buffer (and erased indexes) for now and change later. I'm using the following
logic in HDFS-7678:
{code}
byte[][] decodeInputs = new byte[dataBlkNum + parityBlkNum][]
fetch dataBlkNum blocks into decodeInputs
int[] decodeIndices = new int[parityBlkNum];
for (int i = 0; i < dataBlkNum + parityBlkNum; i++) {
if (decodeInputs[i] == null) {
decodeInputs[i] = new byte[len];
add i to decodeIndices
}
}
rsRawDecoder.initialize(dataBlkNum, parityBlkNum, len)
byte[][] outputs = new byte[parityBlkNum][maxPortion.readLength];
rsRawDecoder.decode(decodeInputs, decodeIndices, outputs);
{code}
After that, you just pick the index you need from {{outputs}}.
I probably won't be able to finish reviewing the whole patch tonight. Will
definite post a full review tomorrow.
> Erasure Coding: striped block recovery
> --------------------------------------
>
> Key: HDFS-7348
> URL: https://issues.apache.org/jira/browse/HDFS-7348
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Kai Zheng
> Assignee: Yi Liu
> Attachments: ECWorker.java, HDFS-7348.001.patch
>
>
> This JIRA is to recover one or more missed striped block in the striped block
> group.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
