[
https://issues.apache.org/jira/browse/HDFS-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Gao updated HDFS-14904:
----------------------------
Description:
Normally the most important purpose for HDFS balancer is to reduce the top used
node to prevent datanode usage from being too high.
Currently, balancer almost randomly picks nodes as sources regardless of usage,
which makes it slow to bring down the top used datanodes in the cluster, when
there are less underutilized nodes in the cluster (consider expansion).
We can add an option to prefer top used nodes first in each iteration, as
suggested in HDFS-14894 .
was:In each iteration, balancer should pick nodes with the highest/lowest
usage first, as suggested in HDFS-14894
Summary: Option to let Balancer prefer top used nodes in each iteration
(was: Balancer should pick nodes based on utilization in each iteration)
> Option to let Balancer prefer top used nodes in each iteration
> --------------------------------------------------------------
>
> Key: HDFS-14904
> URL: https://issues.apache.org/jira/browse/HDFS-14904
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover
> Reporter: Leon Gao
> Assignee: Leon Gao
> Priority: Major
>
> Normally the most important purpose for HDFS balancer is to reduce the top
> used node to prevent datanode usage from being too high.
> Currently, balancer almost randomly picks nodes as sources regardless of
> usage, which makes it slow to bring down the top used datanodes in the
> cluster, when there are less underutilized nodes in the cluster (consider
> expansion).
> We can add an option to prefer top used nodes first in each iteration, as
> suggested in HDFS-14894 .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
