[
https://issues.apache.org/jira/browse/MAPREDUCE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930886#action_12930886
]
dhruba borthakur commented on MAPREDUCE-1608:
---------------------------------------------
Amar, thanks for ur comments. For interactive workloads, this feature has
worked awesomely for us.
> As Todd/Hong have mentioned we should consider making our heuristics better.
> I'm not sure this is a good idea.
I think Arun is completely missing the point here. It is fine to make
heuristics better and better, I have nothing against that. But when there is a
manual-speculate-now button, it can override the system-specified speculation
heuristics for that task. To go a ste further, the system can actually learn
from the user-specified-speculation on how to automatically make the heuristics
better, i.e. crowd-sourcing to the rescue!
> Allow users to do speculative execution of a task manually
> ----------------------------------------------------------
>
> Key: MAPREDUCE-1608
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1608
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Scott Chen
> Assignee: Scott Chen
>
> Speculative execution improves the latency of the job. Sometimes the job has
> few very slow reducers. Spending a little more resource on speculative tasks
> can improve the latency a lot. It will be nice that the users can manually
> select one task and force the speculative execution on that task just like we
> can manually kill/fail task.
> The proposal is add link says "speculate" in taskdetails.jsp page where we do
> "kill/fail".
> Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
