[
https://issues.apache.org/jira/browse/HADOOP-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated HADOOP-4714:
----------------------------------
Status: Open (was: Patch Available)
bq. Attaching a patch that uses a single recordCounter per task. This counter
is used while merging and is not reset per reducer
Passing yet another counter to the merge seems unnecessary; sharing an
AtomicLong between counters to maintain the arbitrary, 10k threshold is
pressing a heuristic into service as an API.
Why not simply report progress at least once for each partition during the
merge?
> map tasks timing out during merge phase
> ---------------------------------------
>
> Key: HADOOP-4714
> URL: https://issues.apache.org/jira/browse/HADOOP-4714
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.18.1
> Reporter: Christian Kunz
> Assignee: Jothi Padmanabhan
> Attachments: hadoop-4714.patch
>
>
> With compression of transient data turned on some parts of the merge phase
> seem to not report progress enough.
> We see a lot of task failures during the merge phase, most of them timing out
> (even with a 20 min timeout)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
