[
https://issues.apache.org/jira/browse/MAPREDUCE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777864#comment-13777864
]
Xuan Gong commented on MAPREDUCE-5533:
--------------------------------------
When the reduce attempt is paused. It will not update its own estimatedRunTime
(because it will process any events that it receives) which will not trigger
the speculative.
The simple method could be :
1. Save the previous estimatedRunTime and progress as history data
2. Compare current estimatedRunTime and current progress with the history data,
if they are the same, we will need to manually kick off the
taskAttemptStatusUpdate.
> Speculative execution does not function for reduce
> --------------------------------------------------
>
> Key: MAPREDUCE-5533
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5533
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Reporter: Tassapol Athiapinya
> Assignee: Xuan Gong
> Fix For: 2.1.1-beta
>
> Attachments: MAPREDUCE-5533.1.patch
>
>
> We have sort job where reduce attempt does not send heartbeat in timely
> manner to application master. The AM should kick off another attempt to let
> job succeeds. What we find is the job fails and there is no speculation
> happening.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
