Ability to turn on/off speculative execution for maps/reduces separately
------------------------------------------------------------------------
Key: HADOOP-2752
URL: https://issues.apache.org/jira/browse/HADOOP-2752
Project: Hadoop Core
Issue Type: New Feature
Components: mapred
Affects Versions: 0.15.3
Reporter: Joydeep Sen Sarma
Motivation is obvious:
- sometimes reduces have side-effects - but maps don't (for example reduces
compute something and write it somewhere)
- want to turn ON speculative for maps (because maps are usually way less
expensive. map progress counters are reliable etc.) - but OFF for reduces (very
expensive to run two reducers instead of one).
It's also likely that in future you may want different setting for speculative
triggers for maps and reduces ... (but that's a different issue).
I will submit a patch (since we are putting this into effect on our cluster -
and it seems simple enough :-)).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
