[
https://issues.apache.org/jira/browse/HDFS-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen He updated HDFS-6159:
--------------------------
Attachment: HDFS-6159-v2.patch
The balancer will create a balancer.id file (about 10B in sieze) in /system
(HDFS) directory when it starts and delete it after it is done. However, this
balancer.id file affects cluster's average utilization if a cluster is made of
datanodes with limited capacities. In the TestBalancerWithNodeGroup class, each
datanode only has 500B total capacity for HDFS and the data block size is 10B.
I propose to increase datanode capacity up to 6000B and data block size to 100B.
Here are the reasons:
1) Increasing datanode capacity can reduce the fluctuation caused by the
balancer.id file;
2) To avoid increasing the number of blocks that the balancer needs to move, I
increase the block size accordingly.
Patch is available for reviewing.
> TestBalancerWithNodeGroup.testBalancerWithNodeGroup fails if there is block
> missing after balancer success
> ----------------------------------------------------------------------------------------------------------
>
> Key: HDFS-6159
> URL: https://issues.apache.org/jira/browse/HDFS-6159
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.3.0
> Reporter: Chen He
> Assignee: Chen He
> Attachments: HDFS-6159-v2.patch, HDFS-6159.patch, logs.txt
>
>
> The TestBalancerWithNodeGroup.testBalancerWithNodeGroup will report negative
> false failure if there is(are) data block(s) losing after balancer
> successfuly finishes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
