[
https://issues.apache.org/jira/browse/HADOOP-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665395#action_12665395
]
Amareshwari Sriramadasu commented on HADOOP-3327:
-------------------------------------------------
Some more numbers:
||Job||With trunk||With patch||
|Sort on 200 nodes|1hrs,1mins,13secs|1hrs, 2mins, 8sec|
|Sort on 200 nodes with read timeouts simulated for 10 maps 5 during the start
of shuffle and 5 during end of shuffle (maxMapRunTime= 3mins, 43sec)| 2hrs,
5mins, 7sec| 1hrs, 5mins, 37sec ( this is almost same as normal run!)|
|SortValidator on 200 nodes with read timeouts for 5 maps during the start of
shuffle(maxMapRunTime = 18mins, 58sec) | 2hrs 13mins 46sec | 31mins, 51sec |
|Gridmix on 200 nodes | 5329 sec | 5187 sec |
|Gridmix on 400 nodes | 3028 sec | 2903 sec |
These results show good improvement incase of fetch failures. And there is no
performance degrading.
> Shuffling fetchers waited too long between map output fetch re-tries
> --------------------------------------------------------------------
>
> Key: HADOOP-3327
> URL: https://issues.apache.org/jira/browse/HADOOP-3327
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Runping Qi
> Assignee: Amareshwari Sriramadasu
> Attachments: hadoop-3327-v1.patch, hadoop-3327-v2.patch,
> hadoop-3327-v3.patch, hadoop-3327.patch, patch-3327.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
