HDFS should blacklist datanodes that are not performing well
------------------------------------------------------------
Key: HADOOP-2956
URL: https://issues.apache.org/jira/browse/HADOOP-2956
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Reporter: dhruba borthakur
On a large cluster, a few datanodes could be under-performing. There were cases
when the network connectivity of a few of these bad datanodes were degraded,
resulting in long long times (in the order of two hours) to transfer blocks to
and from these datanodes.
A similar issue arises when disks a single disk on a datanode fail or change to
read-only mode: in this case the entire datanode shuts down.
HDFS should detect and handle network and disk performance degradation more
gracefully. One option would be to blacklist these datanodes, de-prioritise
their use and alert the administrator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
