[
https://issues.apache.org/jira/browse/MAPREDUCE-6144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223738#comment-14223738
]
Xuan Gong commented on MAPREDUCE-6144:
--------------------------------------
Thanks for working on this. [~zxu]
One question here: do we really need this ? I do not think we need to do this.
Please correct me if I am wrong.
We have SpeculatorEventDispatcher in MRAppMaster which checks the
configurations: MRJobConfig.MAP_SPECULATIVE and MRJobConfig.REDUCE_SPECULATIVE,
If any of the configurations are false, the Map or reduce task will not be
handled by speculator.
And every time when we launched a container, we will send SpeculatorEvent, too.
If the related configuration is set as false, this even will not be handled. In
that case, it will not handle any SpeculatorEvents. We need to handle
SpeculatorEvents to keep the status update. If we do not handle any such
events, when we tries to call computeSpeculations, it will not schedule any
speculations.
> DefaultSpeculator always add both MAP and REDUCE Speculative task even
> MAP_SPECULATIVE or REDUCE_SPECULATIVE is disabled.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6144
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6144
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.5.1
> Reporter: zhihai xu
> Assignee: zhihai xu
> Attachments: MAPREDUCE-6144.000.patch, MAPREDUCE-6144.001.patch
>
>
> DefaultSpeculator always add both MAP and REDUCE Speculative task even
> MAP_SPECULATIVE or REDUCE_SPECULATIVE is disabled.
> If both MAP_SPECULATIVE and REDUCE_SPECULATIVE are disabled,
> DefaultSpeculator won't start.
> The issue will happen if only one of MAP_SPECULATIVE and REDUCE_SPECULATIVE
> is enabled, both MAP and REDUCE Speculative task are generate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
