[
https://issues.apache.org/jira/browse/MAPREDUCE-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu updated MAPREDUCE-4400:
-------------------------------
Attachment: mapreduce-4400-branch-1.patch
MAPREDUCE-2450 fixed the communication race condition that can cause occasional
1 minute timeout but make the PROGRESS_INTERVAL sleep pretty much mandatory.
Any tasks including setup and cleanup tasks would need to sleep at least 3
seconds to finish.
The patch make the wait interruptable when tasks finish.
With this patch and MAPREDUCE-4399 and outofband heartbeats, PigMix is about
20% faster than 0.20.2. These patches makes Hadoop 1.x series on par with
Hadoop 2.x in terms of general performance.
> Fix performance regression for small jobs/workflows
> ---------------------------------------------------
>
> Key: MAPREDUCE-4400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Affects Versions: 0.20.203.0, 1.0.3
> Reporter: Luke Lu
> Assignee: Luke Lu
> Attachments: mapreduce-4400-branch-1.patch
>
>
> There is a significant performance regression for small jobs/workflows (vs
> 0.20.2) in the Hadoop 1.x series. Most noticeable with Hive and Pig jobs.
> PigMix has an average 40% regression against 0.20.2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
