[
https://issues.apache.org/jira/browse/MAPREDUCE-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114383#comment-16114383
]
Peter Bacsko commented on MAPREDUCE-6892:
-----------------------------------------
bq. we can keep the old names and only add new fields, so we continue to use
'finished' as succeeded. In this case, there won't be a new field, succeeded.
You mean we keep the schema but rename stuff in the class from "finished" to
"succeeded", including methods? That's certainly a lot easier.
The names "finishedMaps" and "finishedReduces" occur at other parts of the
code. If you search for either "finishedMaps" or "finishedReduces" in GitHub,
you get 16 results:
https://github.com/apache/hadoop/search?p=1&q=finishedmaps&type=&utf8=%E2%9C%93
Shall we go ahead and do the rename where it's necessary? (there could be some
exceptions like ShuffleSchedulerImpl.java)
bq. 5) In Job20LineHistoryEventEmitter, can we also parse the killed + failed
counters instead of passing -1s?
That's some old code in Apache Rumen. I'm not exactly sure what is parses but I
doubt that would contain string like KILLED_MAPS or KILLED_REDUCES.
bq. 7) In TestJobHistoryParser.testHistoryParsingForKilledAndFailedAttempts(),
verification of noOffailedAttempts seems unnecessary.
That was a copy-paste from an existing testcase
{{testHistoryParsingForFailedAttempts()}}. You sure we don't need that?
> Issues with the count of failed/killed tasks in the jhist file
> --------------------------------------------------------------
>
> Key: MAPREDUCE-6892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6892
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, jobhistoryserver
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Attachments: MAPREDUCE-6892-001.patch, MAPREDUCE-6892-002.PATCH
>
>
> Recently we encountered some issues with the value of failed tasks. After
> parsing the jhist file, {{JobInfo.getFailedMaps()}} returned 0, but actually
> there were failures.
> Another minor thing is that you cannot get the number of killed tasks
> (although this can be calculated).
> The root cause is that {{JobUnsuccessfulCompletionEvent}} contains only the
> successful map/reduce task counts. Number of failed (or killed) tasks are not
> stored.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
