[
https://issues.apache.org/jira/browse/HDFS-8538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573867#comment-14573867
]
Andrew Wang commented on HDFS-8538:
-----------------------------------
Hey Arpit, thanks for taking a look. In the case of imbalance, the
AvailableSpace policy still uses randomness to choose the volume, so it will
still sometimes place on the more full volumes. The writes are skewed towards
the disks with more absolute free space, but not 100% skewed (this skew is
actually configurable). For DNs with same-sized disks, this policy falls back
to RR, so these users are unaffected.
I think a biased round robin based on relative free space as mentioned
[here|https://issues.apache.org/jira/browse/HDFS-1804?focusedCommentId=13623374&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13623374]
would be an improvement, but we've always gotten complaints about uneven disk
usage from customers with differently sized disks, and the current
AvailableSpace policy has worked well in practice.
> Change the default volume choosing policy to
> AvailableSpaceVolumeChoosingPolicy
> -------------------------------------------------------------------------------
>
> Key: HDFS-8538
> URL: https://issues.apache.org/jira/browse/HDFS-8538
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 2.7.0
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Attachments: hdfs-8538.001.patch
>
>
> For datanodes with different sized disks, they almost always want the
> available space policy. Users with homogenous disks are unaffected.
> Since this code has baked for a while, let's change it to be the default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
