[
https://issues.apache.org/jira/browse/HADOOP-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated HADOOP-3586:
----------------------------------
Attachment: 3586-1.patch
No tests are included, as this behavior will change in future releases. The
javac warnings are from the deprecated methods added in support of the
workaround. Fixed the findbugs warning.
> keep combiner backward compatible with earlier versions of hadoop
> -----------------------------------------------------------------
>
> Key: HADOOP-3586
> URL: https://issues.apache.org/jira/browse/HADOOP-3586
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Olga Natkovich
> Assignee: Chris Douglas
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: 3586-0.patch, 3586-1.patch
>
>
> In hadoop 16 and earlier, the combiner was guaranteed to run once and only
> once for each map. In 17 this compatibility was slightly broken: the combiner
> does not run if a single <K,V> occupies the entire sort buffer. In 18, this
> is further changed to where the combiner can be called multiple times on both
> map and reduce sides.
> This breaks Pig's current implementation of the combiner and it is not easy
> to fix in a short period of time.
> We would like to ask that for a way for an application to ask for a backward
> compatible behavior for some period of time until it can adjust to the new
> behavior.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
