[
https://issues.apache.org/jira/browse/MAPREDUCE-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660490#comment-13660490
]
Arun C Murthy commented on MAPREDUCE-4366:
------------------------------------------
Sigh, too late in the night...
bq. No, speculativeMaps == 1, by which we get pendingMaps as 1 since
initialMaps=2 + speculativeMaps=1 - runningMaps=1 - finishedMaps=1 -
failedMaps=0. Correct?
I take it back - that is incorrect since speculativeMaps will be 0, not 1. So
pendingMaps will be wrong till the unnecessary speculative task is killed at
which point we go back to runningMaps = 0 and 'correct' the value for
pendingMaps.
----
I can think of a couple of hacky ways to fix this, but I think I need some
sleep right now.
Maybe the decrement of speculativeTasks shouldn't be done until the cause for
the speculation is marked as KILLED, that is a whole new can of worms. Sigh.
----
Maybe, we should just live with this bug or comb through rest of the codebase
to understand what your original change will need fixing. Nasty.
> mapred metrics shows negative count of waiting maps and reduces
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 1.0.2
> Reporter: Thomas Graves
> Assignee: Sandy Ryza
> Attachments: MAPREDUCE-4366-branch-1-1.patch,
> MAPREDUCE-4366-branch-1.patch
>
>
> Negative waiting_maps and waiting_reduces count is observed in the mapred
> metrics. MAPREDUCE-1238 partially fixed this but it appears there is still
> issues as we are seeing it, but not as bad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
