[
https://issues.apache.org/jira/browse/HDFS-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko updated HDFS-604:
-------------------------------------
Attachment: BlockReportNN.patch
> 1. When a block is under construction, ... NN should only allow to add new
> replicas.
Done.
> 2. NN should not add storedBlock to toCorrupt list if storedBlock' GS is not
> the same as that of the reported replica. Otherwise a wrong GS of the replica
> will get deleted.
This is tricky. I have been back and forth with this. There is a difference
between corrupt and invalid replicas.
_Invalid replica_ is the one that corresponds to a block that is not known to
the name-node and therefore the replica can be safely removed from the
data-node right away.
_Corrupt replicas_ correspond to blocks that exist on the name-name, but the
state of the replica is different from what the name-node expects. This should
include both size and GS.
A corrupt replica cannot be immediately removed from the data-node. First the
name-node should trigger replication, and remove it only when the block is
fully replicated.
All this logic would work fine with my patch if the issue with keys HDFS-512
was resolved. I think we really need it. For now I included temporarily
invalidation of replicas with wrong generation stamp. I plan to remove it and
fix corrupt replica processing, which should remember corrupt replicas with
wrong GS reported by data-nodes.
Will address the safe-mode issue in a separate issue.
> Block report processing for append
> ----------------------------------
>
> Key: HDFS-604
> URL: https://issues.apache.org/jira/browse/HDFS-604
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: Append Branch
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: Append Branch
>
> Attachments: BlockReportNN.patch, BlockReportNN.patch,
> BlockReportNN.patch
>
>
> Implement new block report processing on the name-node as stated in the
> append design and HDFS-576.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
