[
https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004804#comment-16004804
]
Kihwal Lee commented on HDFS-8789:
----------------------------------
What happens on NN restart (to be precise, during the replication queues
initialization)?
Each block will be checked against
{{BlockPlacementPolicy#verifyBlockPlacement()}} and will get added to the
replication queue. When calculating repl work, {{chooseTarget()}} is supposed
to help correct any violation. This happens when the network topology or the
placement policy changes. Does it also work for upgrade domain block placement
policy? It will stress namenode if there are a lot of replicas to be migrated,
but will do the job. I.e. the option number 2 might be already there.
If we are worried about overwhelming NN, we could have a separate thread to do
throttled scan of block placement policy violations. The replication candidate
generation rate would also be throttled. The repl queue init would only deal
with truly under-replicated cases.
> Block Placement policy migrator
> -------------------------------
>
> Key: HDFS-8789
> URL: https://issues.apache.org/jira/browse/HDFS-8789
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Chris Trezzo
> Assignee: Chris Trezzo
> Attachments: HDFS-8789-trunk-STRAWMAN-v1.patch
>
>
> As we start to add new block placement policies to HDFS, it will be necessary
> to have a robust tool that can migrate HDFS blocks between placement
> policies. This jira is for the design and implementation of that tool.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
