Liyin Liang created MAPREDUCE-5691:
--------------------------------------
Summary: Throttle shuffle's bandwidth utilization
Key: MAPREDUCE-5691
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5691
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Liyin Liang
In our hadoop cluster, a reducer of a big job can utilize all the bandwidth
during shuffle phase. Then any task reading data from the machine which
running that reducer becomes very very slow.
It's better to move DataTransferThrottler from hadoop-hdfs to hadoop-common.
And create a throttler for Shuffle to throttle each Fetcher.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
