[
https://issues.apache.org/jira/browse/HADOOP-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507489
]
David Bowen commented on HADOOP-1485:
-------------------------------------
Reviewing 1485.1.patch.
Two classes with the name ShuffleMetrics is confusing. Please rename at least
one of them, and add some per-class comments explaining their purposes.
TaskTracker.ShuffleMetrics:
* shuffle_handler_busy_percent seems to be an absolute value, i.e. it should
be using setMetric rather than incrMetric. Also, shuffle_failed_outputs and
shuffle_success_outputs seem to be relative values, and so should be using
incrMetric rather than setMetric.
* It may be an unnecessary optimization, but it couldn't hurt to move the
shuffleMetricsRecord.update call out of the synchronized block. update() has
to do a little bit of work, and there's no need to be holding the lock.
* MapOutputServlet is missing indentation under the first "try {".
* The final finally may need to call shuffleMetrics.update.
ReduceTask.ReduceCopier.ShuffleMetrics:
* I think incrMetric should be being used for shuffle_failed_fetches and
shuffle_success_fetches.
* Same comment about moving the shuffleMetrics.update() call out of the
synchronized block.
> Metrics should be there for reporting shuffle failures/successes
> ----------------------------------------------------------------
>
> Key: HADOOP-1485
> URL: https://issues.apache.org/jira/browse/HADOOP-1485
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: 1485.1.patch, shuffle-metrics.patch
>
>
> It would be nice to have metrics for the shuffle phase which reports the
> failures/successes for the fetches. This would aid in performance tests and
> in debugging (shuffle).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
