Max Lapan created HDFS-4992:
-------------------------------
Summary: Make balancer's thread count configurable
Key: HDFS-4992
URL: https://issues.apache.org/jira/browse/HDFS-4992
Project: Hadoop HDFS
Issue Type: Improvement
Components: balancer
Reporter: Max Lapan
By default, balancer has 1000 threads which moves blocks around (mover threads)
and 200 threads which decide which block to move (dispatcher threads).
On large clusters, 1000 threads creates significant load on NN, which slows
down other HDFS activity. For example, on our cluster, 'hdfs dfs -ls /' command
took about 1 minute when balancer is active. When no balancing in progress, the
same command finishes in second or two.
This patch makes amount of threads configurable by two new options
'dfs.balancer.moverThreads' and 'dfs.balancer.dispatcherThreads'.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
