[
https://issues.apache.org/jira/browse/HDFS-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934206#comment-14934206
]
Jing Zhao commented on HDFS-7663:
---------------------------------
bq. They have strong relationship when appending is to append a paritial block.
Yes. But lease recovery is a standalone problem and append depends on it. So I
think we can fix lease recovery for EC file first.
bq. Sorry, we are not going to use BlockRecoveryCommand?
It's not we're not going to use BlockRecoveryCommand. What I mean is the
current recovery protocol (i.e., how DN handles the block recovery command)
does not work for an EC block. We need a new protocol similar with what you
proposed in the second part of your doc. Before we have that, we may first want
to disable the recovery for an under construction EC block.
bq. client can overwrite/truncate replica on DN
In our current protocol, data can be truncated when recovering a rbw. More
specifically, when creating a new writing pipeline, a datanode may truncate its
local replica to the safe length (i.e., the length for which the client has
received acks). You can find the code in
{{DataStreamer#createBlockOutputStream}} --> {{DataXceiver#writeBlock}} -->
{{DataXceiver#getBlockReceiver}} --> {{BlockReceiver#BlockReceiver}} -->
{{FsDatasetImpl#recoverRbw}}.
For appending a partial EC block, or hsync/hflush, we can extend existing
protocol or have a new protocol to overwrite the last cell of each parity block.
> Erasure Coding: lease recovery / append on striped file
> -------------------------------------------------------
>
> Key: HDFS-7663
> URL: https://issues.apache.org/jira/browse/HDFS-7663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Walter Su
> Attachments: HDFS-7663.00.txt
>
>
> Append should be easy if we have variable length block support from
> HDFS-3689, i.e., the new data will be appended to a new block. We need to
> revisit whether and how to support appending data to the original last block.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
