[
https://issues.apache.org/jira/browse/MAPREDUCE-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110785#comment-16110785
]
Peter Bacsko edited comment on MAPREDUCE-6892 at 8/2/17 12:28 PM:
------------------------------------------------------------------
[~haibochen] I have been thinking about this renaming and IMO we're better off
solve it in a separate JIRA.
Some problems:
1. I can add new fields to the Avro schema and keep the old ones, but it adds
to the complexity, because we have to support both pairs
2. We probably also have to check whether "finished" or "succeeded" values are
defined in the jhist file, but not both
3. Looking at the {{toTimelineEvent()}} methods, those are a bit messed up too.
As I mentioned above, it has prop. keys like {{NUM_MAPS}} and
{{FINISHED_MAPS}}, but both contain the same value. So either we rename
{{NUM_MAPS}} to {{SUCCEEDED_MAPS}} and delete {{FINISHED_MAPS}} or {{NUM_MAPS}}
should be succeeded+failed+killed and rename {{FINISHED_MAPS}} to
{{SUCCEEDED_MAPS}}.
So it's a bit more complicated. It looks more beneficial to us if we create a
separate JIRA where we try to consolidate the naming inconsistencies (and
possibly involve other upstream devs) and leave these fields alone in this one.
What do you think?
was (Author: pbacsko):
[~haibochen] I have been thinking about this renaming and IMO we're better off
solve it in a separate JIRA.
Some problems:
1. I can add new fields to the Avro schema and keep the old ones, but it adds
to the complexity, because we have to support both pairs
2. We probably also have to check whether "finished" or "succeeded" values are
defined in the jhist file, but not both
3. Looking at the {{toTimelineEvent()}} methods, those are a bit messed up too.
As I mentioned above, it has prop. keys like {{NUM_MAPS}} and
{{FINISHED_MAPS}}, but both contain the same value. So either we rename
{{NUM_MAPS}} to {{SUCCEEDED_MAPS}} and delete {{FINISHED_MAPS}} or {{NUM_MAPS}}
should be succeeded+failed+killed and rename {{FINISHED_MAPS}} to
{{SUCCEEDED_MAPS}}.
So it's a bit more complicated. It looks more beneficial to us if we create a
separate JIRA where we try to consolidate the naming inconsistencies (and
possibly involve other upstream devs) and leave them alone in this one. What do
you think?
> Issues with the count of failed/killed tasks in the jhist file
> --------------------------------------------------------------
>
> Key: MAPREDUCE-6892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6892
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, jobhistoryserver
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Attachments: MAPREDUCE-6892-001.patch, MAPREDUCE-6892-002.PATCH
>
>
> Recently we encountered some issues with the value of failed tasks. After
> parsing the jhist file, {{JobInfo.getFailedMaps()}} returned 0, but actually
> there were failures.
> Another minor thing is that you cannot get the number of killed tasks
> (although this can be calculated).
> The root cause is that {{JobUnsuccessfulCompletionEvent}} contains only the
> successful map/reduce task counts. Number of failed (or killed) tasks are not
> stored.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
