[
https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803316#comment-14803316
]
Zhe Zhang commented on HDFS-9040:
---------------------------------
bq. 3. We might want to add the logic to replace a failed StripedDataStreamer
in the future.
bq. No, we won't. I think so? if you're talking something like Datanode
replacement for repl block. You can transfer a healthy repl RBW to a new
Datanode, then you still get 3 DNs after replacement. But recover a corrupted
RBW internal block is difficult.
I agree it's difficult and in this phase I don't think it's necessary. We
cannot rule out the possibility though. In current non-EC pipeline we support
multiple failover options. A fast writer can opt out in DN replacement and
instead rely on background re-replication. A slow writer might want to replace
DN to prevent data loss during the long window. For a slow EC writer we should
consider fixing the pipeline as well, especially at the early stage of writing
a block (not too much data to decode).
bq. 1. A client read UC block being written can decode replica if it misses
some part. ( With checksum verification, we are only concern about 'missing')
Interesting thought. But {{verifyChecksum}} is optional so we can't always rely
on it. If {{verifyChecksum}} becomes mandatory much of our corrupt replica
handling logic can be much simpler.
> Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests
> to Coordinator)
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-9040
> URL: https://issues.apache.org/jira/browse/HDFS-9040
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Walter Su
> Attachments: HDFS-9040-HDFS-7285.002.patch,
> HDFS-9040-HDFS-7285.003.patch, HDFS-9040.00.patch, HDFS-9040.001.wip.patch,
> HDFS-9040.02.bgstreamer.patch
>
>
> The general idea is to simplify error handling logic.
> Proposal 1:
> A BlockGroupDataStreamer to communicate with NN to allocate/update block, and
> StripedDataStreamer s only have to stream blocks to DNs.
> Proposal 2:
> See below the
> [comment|https://issues.apache.org/jira/browse/HDFS-9040?focusedCommentId=14741388&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14741388]
> from [~jingzhao].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
