[
https://issues.apache.org/jira/browse/HDFS-8328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568107#comment-14568107
]
Zhe Zhang commented on HDFS-8328:
---------------------------------
Thanks for the update Yi! The patch looks good overall. I have the following
questions:
# The use of {{targetsStatus}} is a little tricky. Let me know if I'm
understanding it correctly:
#* If initial connection to a target DN (chosen by NN) is unsuccessful,
{{getErasedIndices}} will not add the index in {{erasedIndices}}; therefore the
block will not be reconstructed. Should we send an ack to NN so the block can
come back to {{UnderReplicatedBlocks}}?
#* In each iteration of {{while (positionInBlock < firstStripedBlockLength)}},
{{targetsStatus}} is reset to reflect if the transfer to each target is
successful. So when an element is flipped from {{true}} to {{false}}, it will
never be flipped back.
#* If the above is correct, maybe let's add some Javadoc and TODO if necessary
(for the NN ack part)?
# {{zeroStripeBuffers}} is created based on whether the entire internal block
is empty. Ideally we should have such an array in each iteration of {{while
(positionInBlock < firstStripedBlockLength)}} so we can avoid issuing read
requests as much as possible. I guess this can be a TODO for follow-on.
# The updated {{scheduleNewRead}} is not so easy to follow.
#* In the first loop, is it assumed that the reader indices in
{{stripedReaders}} begin from 0 and are consecutive?
{code}
int m = stripedReaders.size();
while (reader == null && m < sources.length) {
reader = addStripedReader(m, positionInBlock);
xxx
{code}
#* I feel the method does the right thing but haven't fully understood it yet.
It would be very helpful to have comments for steps 1, 2, 3 in the method.
# The argument and return value of {{readMinimumStripedData4Recovery}} are not
trivial to understand. Maybe add them to the Javadoc?
# Since we are fetching mostly fixed size chunks, it's ideal to reuse buffers
instead of allocating new ones in {{addStripedReader}}. [~walter.k.su] has a
good [analysis |
https://issues.apache.org/jira/browse/HDFS-8481?focusedCommentId=14564053&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14564053]
under HDFS-8481 on the GC issue.
> Follow-on to update decode for DataNode striped blocks reconstruction
> ---------------------------------------------------------------------
>
> Key: HDFS-8328
> URL: https://issues.apache.org/jira/browse/HDFS-8328
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Yi Liu
> Assignee: Yi Liu
> Attachments: HDFS-8328-HDFS-7285.001.patch,
> HDFS-8328-HDFS-7285.002.patch
>
>
> Current the decode for DataNode striped blocks reconstruction is a
> workaround, we need to update it after the decode fix in HADOOP-11847.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
