[
https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569792#comment-14569792
]
Zhe Zhang commented on HDFS-8319:
---------------------------------
Great work Jing! {{StripeReader}} is a very good abstraction for p/np reads.
The patch looks pretty good to me. A few thoughts are listed below; some are
suggested future work.
# Nit: we can move {{readToBuffer}} to be right after {{readCell}}, which is
its only caller.
# We can rename {{prepareParityChunk}} to {{prepareAdditionalChunks}} because
in pread we will possibly issue additional data chunks to be fetched. Then we
can move out the chunk-prep code from {{initDecodeInputs}} either in this patch
or as a follow-on (there are a few other places where we can further
consolidate the code anyway).
# In both read modes, our recovery algorithm is to issue all possible requests
upon _first failure_. We can think about if we want to support other less
expensive algorithms. If not, we can change the structure of the while loop in
{{readStripe}} so that we don't call {{prepareDecodeInputs}} and
{{prepareParityChunk}} for each missing block.
# {{DFSStripedInputStream#copy}} could perhaps use a better name. The Javadoc
is pretty clear though.
# We should either rename {{testWritePreadWithDNFailure}} or have the stateful
read as a separate test. Rename to {{testWriteReadWithDNFailure}} is easier.
BTW the test failure is caused by the order of parity and data blocks in the
decode input buffers (see HDFS-8517).
# As a follow-on, we can probably always convert byte array to byte buffer so
we can further consolidate {{divideOneStripe}} with
{{divideByteRangeIntoStripes}} and the internal code of {{StripingChunk}};
maybe we can also consolidate the 2 {{decode()}}.
# As a follow-on, we can consider making {{StripeReader}} static so DataNode
can potentially use it too.
> Erasure Coding: support decoding for stateful read
> --------------------------------------------------
>
> Key: HDFS-8319
> URL: https://issues.apache.org/jira/browse/HDFS-8319
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-8319.001.patch, HDFS-8319.002.patch
>
>
> HDFS-7678 adds the decoding functionality for pread. This jira plans to add
> decoding to stateful read.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
