DFS should detect slow links(nodes) and avoid them
--------------------------------------------------
Key: HADOOP-3589
URL: https://issues.apache.org/jira/browse/HADOOP-3589
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Reporter: Runping Qi
The current DFS does not detect slow links (nodes).
Thus, when a node or its network link is slow, it may affect the overall system
performance significantly.
Specifically, when a map job needs to read data from such a node, it may
progress 10X slower.
And when a DFS data node pipeline consists of such a node, the write
performance degrades significantly.
This may lead to some long tails for map/reduce jobs. We have experienced such
behaviors quite often.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
