[
https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805014#comment-14805014
]
Li Bo commented on HDFS-9040:
-----------------------------
Thanks for the deep discussion here. One point confused me:
bq. Only based on internal block lengths we cannot identify the failure
For a block not belong to the last block group, if its length is smaller than
BLOCK_SIZE, we can conclude it’s corrupt, otherwise it is a good block, right?
For the last block group, we can calculate the length of each block via the
file length, so if a block doesn’t satisfy the required length, then we can
conclude it’s corrupt. The precondition is, if <=NUM_PARITY streamers fail, we
ignore their failures and treat all blocks of this block group are written
correctly.
Each block in a block group only has one replica, and if we can judge a block
corrupt or not, it may not necessary to bump the GS.
Any other points that require bumping the GS?
> Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests
> to Coordinator)
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-9040
> URL: https://issues.apache.org/jira/browse/HDFS-9040
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Walter Su
> Attachments: HDFS-9040-HDFS-7285.002.patch,
> HDFS-9040-HDFS-7285.003.patch, HDFS-9040.00.patch, HDFS-9040.001.wip.patch,
> HDFS-9040.02.bgstreamer.patch
>
>
> The general idea is to simplify error handling logic.
> Proposal 1:
> A BlockGroupDataStreamer to communicate with NN to allocate/update block, and
> StripedDataStreamer s only have to stream blocks to DNs.
> Proposal 2:
> See below the
> [comment|https://issues.apache.org/jira/browse/HDFS-9040?focusedCommentId=14741388&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14741388]
> from [~jingzhao].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
