[
https://issues.apache.org/jira/browse/HDFS-9007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-9007:
--------------------------
Attachment: HDFS-9007.patch
Here is the initial patch. The basic idea is to add a new method called
{{isMovable}} to {{BlockPlacementPolicy}} for balancer and other tools to ask
if the move is allowed.
The reason we don't use {{verifyBlockPlacement}} in such scenario is
{{isMovable}} and {{verifyBlockPlacement}} have different meanings.
{{isMovable}} means "it is ok as long as it doesn't make things worse.". For
example, say a block has two replicas and they are on the same rack. Moving one
of the replicas to another node in the same rack is allowed from balancer's
point of view. But {{verifyBlockPlacement}} won't allow that.
The policy defined by {{isMovable}} is used by client-side tools. It is the
same as the policy used by BlockManager to decide delete hint. Thus there is
some refactoring to make sure that policy is shared properly.
The patch also adds bunch of new tests and clean up some code along the way.
> Fix HDFS Balancer to honor upgrade domain policy
> ------------------------------------------------
>
> Key: HDFS-9007
> URL: https://issues.apache.org/jira/browse/HDFS-9007
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ming Ma
> Attachments: HDFS-9007.patch
>
>
> In the current design of HDFS Balancer, it doesn't use BlockPlacementPolicy
> used by namenode runtime. Instead, it has somewhat redundant code to make
> sure block allocation conforms with the rack policy.
> When namenode uses upgrade domain based policy, we need to make sure that
> HDFS balancer doesn't move blocks in a way that could violate upgrade domain
> block placement policy.
> In the longer term, we should consider how to make Balancer independent of
> the actual BlockPlacementPolicy as in HDFS-1431.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
