[
https://issues.apache.org/jira/browse/MAPREDUCE-5691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Liang updated MAPREDUCE-5691:
-----------------------------------
Attachment: ganglia-slave.jpg
Attach a ganglia network metics picture. The reducer utilizes all the input
bandwidth. So the throttling should go on the reducer side.
> Throttle shuffle's bandwidth utilization
> ----------------------------------------
>
> Key: MAPREDUCE-5691
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5691
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.2.0
> Reporter: Liyin Liang
> Attachments: ganglia-slave.jpg
>
>
> In our hadoop cluster, a reducer of a big job can utilize all the bandwidth
> during shuffle phase. Then any task reading data from the machine which
> running that reducer becomes very very slow.
> It's better to move DataTransferThrottler from hadoop-hdfs to hadoop-common.
> And create a throttler for Shuffle to throttle each Fetcher.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
