[
https://issues.apache.org/jira/browse/HDFS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer resolved HDFS-284.
-----------------------------------
Resolution: Won't Fix
> dfs.data.dir syntax needs revamping: multiple percentages and weights
> ---------------------------------------------------------------------
>
> Key: HDFS-284
> URL: https://issues.apache.org/jira/browse/HDFS-284
> Project: Hadoop HDFS
> Issue Type: Improvement
> Environment: This is likely a cross-platform issue.
> Reporter: Allen Wittenauer
> Priority: Minor
>
> Currently, all filesystems listed in the dfs.data.dir are treated the same
> with respected to the space reservation percentages. This makes sense on
> homogeneous, dedicated machines, but breaks badly on heterogeneous ones and
> creates a bit of a support nightmare.
> In a grid with multiple disk sizes, the admin is either leaving space
> unallocated or is required to slice up the disk. In addition, if Hadoop
> isn't the only application running, there may be unexpected collisions. In
> order to work around this limitation, the administrator must specifically
> partition up filesystem space such that the reservation 'make sense' for all
> of the configured file systems. For example, if someone has 2 small file
> systems and 2 big ones on a single machine, due to various requirements (such
> as the OS being mirrored, systems were built from spare parts, server
> consolidation, whatever). Reserving 10% might make sense on the small file
> systems (say 7G) but 10% may leave a lot more space than desired free on the
> big ones (say 50G).
> Instead, Hadoop should support a more robust syntax for directory layout.
> Ideally, an admin should be able to specify the directory location, the
> amount of space reserved (in either a percentage or a raw size syntax) for
> HDFS, as well as a weighting such that some file systems may be preferred
> over others. In the example above, the two larger file systems would likely
> be preferred over the two smaller ones. Additionally, the reservation on the
> larger file system might be changed such that it matches the 7G on the
> smaller file system.
> Doing so would allow for much more complex configuration scenarios without
> having to shuffle a lot of things around at the operating system level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
