[
https://issues.apache.org/jira/browse/HDFS-339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved HDFS-339.
--------------------------
Resolution: Not A Problem
The introduction of the DFS Balancer (post '06, apparently) provides the
feature of moving blocks around and balancing the DFS DNs.
I personally do not think its a good idea to add a monitor to the NN for
auto-triggering balancers since it would use up bandwidth without the
user/admin ever knowing about it. One could surely write external tools to
achieve this monitoring and running separately though.
Resolving a Not-A-Problem, but do reopen if you feel strongly that the NN would
really benefit from an additional service as this.
> Periodically move blocks from full nodes to those with space
> -------------------------------------------------------------
>
> Key: HDFS-339
> URL: https://issues.apache.org/jira/browse/HDFS-339
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Bryan Pendleton
> Assignee: Sameer Paranjpye
>
> Continuance of Hadoop-386. The patch to that issue makes it possible to
> redistribute blocks (change replication up, wait for replication to succeed,
> then lower replication again). However, this requires a lot more space, is
> not automatic, and doesn't respect a reasonable I/O limit. I have actually
> had MapReduce jobs fail from block missing execptions after having recently
> changed the replication level (from 3 to 4, with no underreplications to
> start with) because the datanodes were too slow responding to requests while
> performing the necessary replications.
> A good fix to this problem would be a low-priority thread on the NameNode
> that schedules low-priority replications of blocks on over-full machines,
> followed by the removal of the extra replications. It might be worth having a
> specific prototocol for asking for these low-priority copies to happen in the
> datanodes, so that they continue to service (and be available to service)
> normal block requests.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
