[
https://issues.apache.org/jira/browse/MAPREDUCE-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728350#action_12728350
]
Matei Zaharia commented on MAPREDUCE-725:
-----------------------------------------
In the fair scheduler, my plan was to look at speculative task counts every
10-15 seconds, because while the count is dynamic, speculation decisions are
made on the order of minutes anyway. Therefore, I'd be okay with a
numSpeculative{Maps|Reduces} that is time-consuming to compute and may change
fairly rapidly. However, if you don't think this should be part of the API,
then just hasSpeculative{Map|Reduce} is fine. I'll try making just one
speculative slot at a time and see if that causes problems.
> CapacityScheduler.TaskSchedulingMgr.hasSpeculativeTask bypasses HADOOP-2141
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-725
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-725
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/capacity-sched
> Reporter: Arun C Murthy
> Priority: Critical
>
> CapacityScheduler.TaskSchedulingMgr.hasSpeculativeTask has a duplicate of the
> old speculation code (pre HADOOP-2141) which needs to be fixed in-order for
> speculation to work correctly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
