[
https://issues.apache.org/jira/browse/MAPREDUCE-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das updated MAPREDUCE-717:
----------------------------------
Attachment: 717.patch
Apart from what I mentioned in the jira description, here are a few other fixes:
1) Changed the speculative cap to take into account the tasktype (map and
reduce). In the HADOOP-2141 patch, it is a global count for maps and reduces.
The problem I encountered was that in some cases, the reduce tasks would take
up the slots towards the beginning of the job, and the last few maps wouldn't
get any speculative slot.
2) Changed the dispatch time for a task to be per task-attempt based. In the
HADOOP-2141 patch, it is per task based.
All the unit tests & test-patch passed.
> Fix some corner case issues in speculative execution (post hadoop-2141)
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-717
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-717
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.21.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.21.0
>
> Attachments: 717.patch, 717.patch
>
>
> Some corner case issues can be fixed:
> 1) Setup task should not add anything to the job statistics (since they are
> really fast and might affect the statistics of a job with few tasks)
> 2) The statistics computations should be guarded for cases where things like
> sumOfSquares could become less than zero (due to rounding errors mostly).
> 3) The method TaskInProgress.getCurrentProgressRate() should take into
> account the COMMIT_PENDING state
> 4) The testcase TestSpeculativeExecution.testTaskLATEScheduling could be made
> more robust
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
