[
https://issues.apache.org/jira/browse/HDFS-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191865#comment-13191865
]
Todd Lipcon commented on HDFS-2791:
-----------------------------------
The code path calls {{markBlockAsCorrupt}} which adds it to
{{corruptReplicasMap}}. This will cause another replica to get made, and then
the original will be deleted -- wasted work but not the end of the world.
In the non-HA case, this is probably OK. But in the HA case, this race can
happen even when a file's replication count is 1, in which case it will get
marked corrupt and then no new copy can be made. The block will be corrupt
until the next full block report.
> If block report races with closing of file, replica is incorrectly marked
> corrupt
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2791
> URL: https://issues.apache.org/jira/browse/HDFS-2791
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-2791-test.txt, hdfs-2791.txt, hdfs-2791.txt,
> hdfs-2791.txt
>
>
> The following sequence of events results in a replica mistakenly marked
> corrupt:
> 1. Pipeline is open with 2 replicas
> 2. DN1 generates a block report but is slow in sending to the NN (eg some
> flaky network). It gets "stuck" right before the block report RPC.
> 3. Client closes the file.
> 4. DN2 is fast and sends blockReceived to the NN. NN marks the block as
> COMPLETE
> 5. DN1's block report proceeds, and includes the block in an RBW state.
> 6. (x) NN incorrectly marks the replica as corrupt, since it is an RBW
> replica on a COMPLETE block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
