For my education, can someone tell:
was "combiner guaranteed to run once and only once for each map" a
documented part of the contract ?
-- ab
On Jun 17, 2008, at 3:28 PM, Olga Natkovich (JIRA) wrote:
keep combiner backward compatible with earlier versions of hadoop
-----------------------------------------------------------------
Key: HADOOP-3586
URL: https://issues.apache.org/jira/browse/
HADOOP-3586
Project: Hadoop Core
Issue Type: Bug
Reporter: Olga Natkovich
Fix For: 0.18.0
In hadoop 16 and earlier, the combiner was guaranteed to run once
and only once for each map. In 17 this compatibility was slightly
broken: the combiner does not run if a single <K,V> occupies the
entire sort buffer. In 18, this is further changed to where the
combiner can be called multiple times on both map and reduce sides.
This breaks Pig's current implementation of the combiner and it is
not easy to fix in a short period of time.
We would like to ask that for a way for an application to ask for a
backward compatible behavior for some period of time until it can
adjust to the new behavior.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
