[
https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000699#comment-14000699
]
Ming Ma commented on HDFS-6425:
-------------------------------
Sorry, the behavior was based on early version of hadoop. I just checked the
trunk. It seems even after active is transitioned to standby,
postponedMisreplicatedBlocks will reduce over time as DNs blockReport come in.
Is it possible when a larger postponedMisreplicatedBlocks makes each
blockReport takes longer time, it will make the cluster generate more over
replica due to NN lock contention?
> reset postponedMisreplicatedBlocks and postponedMisreplicatedBlocksCount when
> NN becomes active
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-6425
> URL: https://issues.apache.org/jira/browse/HDFS-6425
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: HDFS-6425.patch
>
>
> Sometimes we have large number of over replicates when NN fails over. When
> the new active NN took over, over replicated blocks will be put to
> postponedMisreplicatedBlocks until all DNs for that block aren't stale
> anymore.
> We have a case where NNs flip flop. Before postponedMisreplicatedBlocks
> became empty, NN fail over again and again. So postponedMisreplicatedBlocks
> just kept increasing until the cluster is stable.
> In addition, large postponedMisreplicatedBlocks could make
> rescanPostponedMisreplicatedBlocks slow. rescanPostponedMisreplicatedBlocks
> takes write lock. So it could slow down the block report processing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
