[
https://issues.apache.org/jira/browse/HDFS-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289795#comment-14289795
]
Aaron T. Myers commented on HDFS-7421:
--------------------------------------
Hey Kihwal, yes indeed, this seems like a dupe. I'll go ahead and close this
one. Thanks for pointing that out, and thanks for filing/fixing the issue in
HDFS-6425.
> Move processing of postponed over-replicated blocks to a background task
> ------------------------------------------------------------------------
>
> Key: HDFS-7421
> URL: https://issues.apache.org/jira/browse/HDFS-7421
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha, namenode
> Affects Versions: 2.6.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
>
> In an HA environment, we postpone sending block invalidates to DNs until all
> DNs holding a given block have done at least one block report to the NN after
> it became active. When that first block report after becoming active does
> occur, we attempt to reprocess all postponed misreplicated blocks inline with
> the block report RPC. In the case where there are many postponed
> misreplicated blocks, this can cause block report RPCs to take an
> inordinately long time to complete, sometimes on the order of minutes, which
> has the potential to tie up RPC handlers, block incoming RPCs, etc. There's
> no need to hurriedly process all postponed misreplicated blocks so that we
> can quickly send invalidate commands back to DNs, so let's move this
> processing outside of the RPC handler context and into a background thread.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
