[
https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791655#comment-14791655
]
Zhe Zhang commented on HDFS-9040:
---------------------------------
Thanks Jing for the new patch! The structure looks much cleaner now.
I've been thinking about the design to check streamer failures at
{{writeChunk}} and other events on {{OutputStream}} level. The code structure
is certainly simpler than handling failures on streamer level. But are there
any disadvantages to delay the handling of a streamer failure? If there isn't
any downside, should we just do {{updatePipeline}} when completing the block?
A few possible disadvantages I can think of:
# In the read-being-written scenario, there will be a longer window of
*false-fresh" (meaning a stale internal block is considered as fresh).
# When {{NUM_PARITY_BLOCKS}} number of streamers are dead, the {{OutputStream}}
should die immediately instead of waiting for the next {{writeChunk}}.
# We might want to add the logic to replace a failed {{StripedDataStreamer}} in
the future. Delayed error handling will cause delayed streamer replacement.
> Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests
> to Coordinator)
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-9040
> URL: https://issues.apache.org/jira/browse/HDFS-9040
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Walter Su
> Attachments: HDFS-9040-HDFS-7285.002.patch,
> HDFS-9040-HDFS-7285.003.patch, HDFS-9040.00.patch, HDFS-9040.001.wip.patch,
> HDFS-9040.02.bgstreamer.patch
>
>
> The general idea is to simplify error handling logic.
> Proposal 1:
> A BlockGroupDataStreamer to communicate with NN to allocate/update block, and
> StripedDataStreamer s only have to stream blocks to DNs.
> Proposal 2:
> See below the
> [comment|https://issues.apache.org/jira/browse/HDFS-9040?focusedCommentId=14741388&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14741388]
> from [~jingzhao].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
