[
https://issues.apache.org/jira/browse/MAPREDUCE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920026#action_12920026
]
Chris Douglas commented on MAPREDUCE-2124:
------------------------------------------
bq. These times are recorded in job history but it will be nice to have them on
the web UI if there is no much cost.
OK, I see; I thought the proposal was to add a task counter. Is this intended
for the job summary log or for feedback to users? I guess one could consider
the shuffle bytes and these times to help tune their job... OK, I think I
understand. Thanks for explaining. Should this ignore failed/killed tasks? e.g.
speculatively executed reduces killed in the shuffle stage probably shouldn't
add to the accumulator.
> Add job counters for measuring time spent in three different phases in
> reducers
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2124
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2124.txt
>
>
> We currently have SLOTS_MILLIS_REDUCES which measures the total slot time of
> reducer.
> It will be useful if we have
> {code}
> SLOTS_MILLIS_REDUCES_COPY
> SLOTS_MILLIS_REDUCES_SORT
> SLOTS_MILLIS_REDUCES_REDUCE
> {code}
> which measures three different phases of a reducer.
> This will help us identify the bottleneck of the reducers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
