[
https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911798#comment-16911798
]
Wei-Chiu Chuang edited comment on HDFS-8789 at 8/20/19 11:06 PM:
-----------------------------------------------------------------
Very quickly went through the patch.
(1) if this tool runs for an extended period of time, say more than a day, it
may fail in a Kerberized environment since it does not renew kerberos
credentials. It doesn't renew block keys. Just like HDFS-11741.
(2) I'm still not clear about the process of running this tool -- is an
administrator supposed to run this tool before restarting NN and enabling
BlockPlacementPolicyWithUpgradeDomain? But NameNode would treat the new
replica as mis-replicated block replica, and recalculate, wouldn't it? The
migrator has to finish migration faster than NameNode can move replicas back.
(3) What if I simply skip the migrator step, and enable UD. NameNode would see
a huge number of mis-replicated blocks, but it should eventually move the
replicas to the correct location. Of course, we need to fix HDFS-14637 first.
(4) i wonder if it makes sense to integrate migrator to balancer/mover. There
are already balancer/mover/diskbalancer. Having another similar tool in the
arsenal can be a support burden. From what I see, balancer doesn't support
anything other than the default placement policy (BlockPlacementPolicyDefault).
Is the migrator tool sort of like a balancer designed to support UD?
was (Author: jojochuang):
Very quickly went through the patch.
(1) if this tool runs for an extended period of time, say more than a day, it
may fail in a Kerberized environment since it does not renew kerberos
credentials. Just like HDFS-11741.
(2) I'm still not clear about the process of running this tool -- is an
administrator supposed to run this tool before restarting NN and enabling
BlockPlacementPolicyWithUpgradeDomain? But NameNode would treat the new
replica as mis-replicated block replica, and recalculate, wouldn't it? The
migrator has to finish migration faster than NameNode can move replicas back.
(3) What if I simply skip the migrator step, and enable UD. NameNode would see
a huge number of mis-replicated blocks, but it should eventually move the
replicas to the correct location. Of course, we need to fix HDFS-14637 first.
(4) i wonder if it makes sense to integrate migrator to balancer/mover. There
are already balancer/mover/diskbalancer. Having another similar tool in the
arsenal can be a support burden. From what I see, balancer doesn't support
anything other than the default placement policy (BlockPlacementPolicyDefault).
Is the migrator tool sort of like a balancer designed to support UD?
> Block Placement policy migrator
> -------------------------------
>
> Key: HDFS-8789
> URL: https://issues.apache.org/jira/browse/HDFS-8789
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Chris Trezzo
> Assignee: Chris Trezzo
> Priority: Major
> Attachments: HDFS-8789-trunk-STRAWMAN-v1.patch
>
>
> As we start to add new block placement policies to HDFS, it will be necessary
> to have a robust tool that can migrate HDFS blocks between placement
> policies. This jira is for the design and implementation of that tool.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
