[
https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092796#comment-15092796
]
Anu Engineer commented on HDFS-1312:
------------------------------------
Hi [~andrew.wang],
As discussed off-line, let us meet on 14th of Jan, 2016 @ 4:00 - 5:00 PM PST.
Here is the meeting info.
I look forward to chatting with other Apache members who might be interested in
this topic.
Anu Engineer is inviting you to a scheduled Zoom meeting.
{noformat}
Topic: HDFS-1312 discussion
Time: Jan 14, 2016 4:00 PM (GMT-8:00) Pacific Time (US and Canada)
Join from PC, Mac, Linux, iOS or Android:
https://hortonworks.zoom.us/j/267578285
Or join by phone:
+1 646 558 8656 (US Toll) or +1 408 638 0968 (US Toll)
+1 855 880 1246 (US Toll Free)
+1 888 974 9888 (US Toll Free)
Meeting ID: 267 578 285
International numbers available:
https://hortonworks.zoom.us/zoomconference?m=ZlHRHTGmVEKXzM_RaCyzcSnjlk_z3ovm
{noformat}
> Re-balance disks within a Datanode
> ----------------------------------
>
> Key: HDFS-1312
> URL: https://issues.apache.org/jira/browse/HDFS-1312
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode
> Reporter: Travis Crawford
> Assignee: Anu Engineer
> Attachments: Architecture_and_testplan.pdf, disk-balancer-proposal.pdf
>
>
> Filing this issue in response to ``full disk woes`` on hdfs-user.
> Datanodes fill their storage directories unevenly, leading to situations
> where certain disks are full while others are significantly less used. Users
> at many different sites have experienced this issue, and HDFS administrators
> are taking steps like:
> - Manually rebalancing blocks in storage directories
> - Decomissioning nodes & later readding them
> There's a tradeoff between making use of all available spindles, and filling
> disks at the sameish rate. Possible solutions include:
> - Weighting less-used disks heavier when placing new blocks on the datanode.
> In write-heavy environments this will still make use of all spindles,
> equalizing disk use over time.
> - Rebalancing blocks locally. This would help equalize disk use as disks are
> added/replaced in older cluster nodes.
> Datanodes should actively manage their local disk so operator intervention is
> not needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
