[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994840#comment-15994840
]
Kihwal Lee commented on HDFS-8818:
----------------------------------
We see this problem in EVERY single cluster we tried this on.
The first big three bumps (20 min each) is 2.8 with this change reverted. The
following small ones are the result of non-reverted 2.8 with the hang fix
(HDFS-11377). It is still no where near what it used to be.
!https://issues.apache.org/jira/secure/attachment/12866172/bal1.png!
This is the stock 2.8 balancer without HDFS-11377.
!https://issues.apache.org/jira/secure/attachment/12866171/bal2.png!
I don't doubt that it worked great for the case you designed and tested for.
However, please do realize that it will be a regression for many other users.
I am not denying the shortcomings of the existing design. But the new design
has clear issues, which cannot be worked around with config tweaks. Even if it
works better for some cases, it is a regression.
> Allow Balancer to run faster
> ----------------------------
>
> Key: HDFS-8818
> URL: https://issues.apache.org/jira/browse/HDFS-8818
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: balancer & mover
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
> Attachments: bal1.png, bal2.png, h8818_20150723.patch,
> h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch
>
>
> The original design of Balancer is intentionally to make it run slowly so
> that the balancing activities won't affect the normal cluster activities and
> the running jobs.
> There are new use case that cluster admin may choose to balance the cluster
> when the cluster load is low, or in a maintain window. So that we should
> have an option to allow Balancer to run faster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
