[
https://issues.apache.org/jira/browse/HDFS-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246874#comment-17246874
]
Ayush Saxena commented on HDFS-15170:
-------------------------------------
IIRC In case of EC blocks, if one block is corrupt, the block group was getting
marked as corrupt. In BlockManager :
{code:java}
// Add this replica to corruptReplicas Map. For striped blocks, we always
// use the id of whole striped block group when adding to corruptReplicas
Block corrupted = new Block(b.getCorrupted());
if (b.getStored().isStriped()) {
corrupted.setBlockId(b.getStored().getBlockId());
}
{code}
Well to be precise, I don't remember exactly what happened because of that(I
don't have access to the internal discussion now) and I guess Surendra was
involved in this with me, He too won't be having access, This came from some
testing side, Don't remember how we landed up at HDFS-15200. Will try catch up
with folks, if anyone remembers what exactly happened.
> EC: Block gets marked as CORRUPT in case of failover and pipeline recovery
> --------------------------------------------------------------------------
>
> Key: HDFS-15170
> URL: https://issues.apache.org/jira/browse/HDFS-15170
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Critical
> Attachments: HDFS-15170-01.patch, HDFS-15170-02.patch,
> HDFS-15170-03.patch
>
>
> Steps to Repro :
> 1. Start writing a EC file.
> 2. After more than one stripe has been written, stop one datanode.
> 3. Post pipeline recovery, keep on writing the data.
> 4.Close the file.
> 5. transition the namenode to standby and back to active.
> 6. Turn on the shutdown datanode in step 2
> The BR from datanode 2 will make the block corrupt and during invalidate
> block won't remove it, since post failover the blocks would be on stale
> storage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
