[
https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Chansler updated HADOOP-2141:
------------------------------------
Release Note: (was: Improves the speculative execution heuristic. The
heuristic is currently based on the progress-rates of tasks and the expected
time of completion. Also, statistics about trackers are collected, and
speculative tasks are not given to the ones deduced to be slow.)
Editorial pass over all release notes prior to publication of 0.21. Routine.
> speculative execution start up condition based on completion time
> -----------------------------------------------------------------
>
> Key: HADOOP-2141
> URL: https://issues.apache.org/jira/browse/HADOOP-2141
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.21.0
> Reporter: Koji Noguchi
> Assignee: Andy Konwinski
> Fix For: 0.21.0
>
> Attachments: 2141.4.patch, 2141.5.patch, 2141.6.patch, 2141.7.patch,
> 2141.8.2.patch, 2141.8.3.patch, 2141.patch, HADOOP-2141-v2.patch,
> HADOOP-2141-v3.patch, HADOOP-2141-v4.patch, HADOOP-2141-v5.patch,
> HADOOP-2141-v6.patch, HADOOP-2141.patch, HADOOP-2141.v7.patch,
> HADOOP-2141.v8.patch
>
>
> We had one job with speculative execution hang.
> 4 reduce tasks were stuck with 95% completion because of a bad disk.
> Devaraj pointed out
> bq . One of the conditions that must be met for launching a speculative
> instance of a task is that it must be at least 20% behind the average
> progress, and this is not true here.
> It would be nice if speculative execution also starts up when tasks stop
> making progress.
> Devaraj suggested
> bq. Maybe, we should introduce a condition for average completion time for
> tasks in the speculative execution check.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
