[
https://issues.apache.org/jira/browse/HDFS-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744191#comment-14744191
]
Zhe Zhang commented on HDFS-8889:
-
Thanks for the work Bo. It's a great idea to test the write pipeline error
handling more systematically. I just moved this JIRA to follow-on together with
other write pipeline JIRAs.
> Erasure Coding: cover more test situations of datanode failure during client
> writing
>
>
> Key: HDFS-8889
> URL: https://issues.apache.org/jira/browse/HDFS-8889
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Li Bo
>Assignee: Li Bo
> Attachments: HDFS-8889-HDFS-7285-001.patch
>
>
> Currently 9 streamers are working together for the client writing. A small
> number of failed datanodes (<= 3) for a block group should not influence the
> writing. There’re a lot of datanode failure cases and we should cover as many
> as possible in unit test.
> Suppose streamer 4 fails, the following situations for the next block group
> should be considered:
> 1)all streamers succeed
> 2)Streamer 4 still fails
> 3)only streamer 1 fails
> 4)only streamer 8 fails (test parity streamer)
> 5)streamer 4 and 6 fail
> 6)streamer 4 and 1,6 fail
> 7)streamer 4 and 1,2,6 fail
> 8)streamer 2, 6 fail
> Suppose streamer 2 and 4 fail, the following situations for the next block
> group should be considered:
> 1)only streamer 2 and 4 fail
> 2)streamer 2, 4, 8 fail
> 3)only streamer 2 fails
> 4)streamer 3 , 8 fail
> For a single streamer, we should consider the following situations of the
> time of datanode failure:
> 1)before writing the first byte
> 2)before finishing writing the first cell
> 3)right after finishing writing the first cell
> 4)before writing the last byte of the block
> Other situations:
> 1)more than 3 streamers fail at the first block group
> 2)more than 3 streamers fail at the last block group
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
