[
https://issues.apache.org/jira/browse/MAPREDUCE-479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Tan updated MAPREDUCE-479:
--------------------------------
Release Note: Adds Reduce Attempt ID to ClientTrace log messages, and adds
Reduce Attempt ID to HTTP query string sent to mapOutputServlet. Extracts
partition number from attempt ID. (was: Adds Reduce Attempt ID to ClientTrace
log messages, and adds Reduce Attempt ID to HTTP query string sent to
mapOutputServlet.)
Status: Patch Available (was: Open)
Did microbenchmark of shuffle durations with and without added reduce attempt
ID transmission and reduce partition number extraction; shuffle times before
and after this patch are statistically comparable (chi-squared test for
distribution similarity of shuffle times, p-value 0.23 => null-hypothesis of
statistically different distributions not rejected); thus this patch does not
cause any performance impact.
> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
> Key: MAPREDUCE-479
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-479
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.21.0
> Reporter: Jiaqi Tan
> Assignee: Jiaqi Tan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-6013.patch, MAPREDUCE-479-1.patch,
> MAPREDUCE-479-2.patch, MAPREDUCE-479-3.patch, MAPREDUCE-479-4.patch,
> MAPREDUCE-479.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID
> but not the source reduce ID. Having both source and destination ID of each
> shuffle enables full tracing of execution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
