[
https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated HDFS-6425:
--------------------------------------
Component/s: namenode
> Large postponedMisreplicatedBlocks has impact on blockReport latency
> --------------------------------------------------------------------
>
> Key: HDFS-6425
> URL: https://issues.apache.org/jira/browse/HDFS-6425
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Ming Ma
> Assignee: Ming Ma
> Fix For: 2.7.0
>
> Attachments: HDFS-6425-2.patch, HDFS-6425-3.patch,
> HDFS-6425-Test-Case.pdf, HDFS-6425.patch
>
>
> Sometimes we have large number of over replicates when NN fails over. When
> the new active NN took over, over replicated blocks will be put to
> postponedMisreplicatedBlocks until all DNs for that block aren't stale
> anymore.
> We have a case where NNs flip flop. Before postponedMisreplicatedBlocks
> became empty, NN fail over again and again. So postponedMisreplicatedBlocks
> just kept increasing until the cluster is stable.
> In addition, large postponedMisreplicatedBlocks could make
> rescanPostponedMisreplicatedBlocks slow. rescanPostponedMisreplicatedBlocks
> takes write lock. So it could slow down the block report processing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
