Right now Hive does not exploit the combiner. But hash-based map-side aggregation in hive (controlled by hints) provides a similar optimization. Using the combiner in addition to map-side aggregation should improve the performance even more if the combiner can further aggregate the partial aggregates generated from the mapper.
On 2/26/09 5:57 AM, "Qing Yan" <[email protected]> wrote: > Is there any way/plan for Hive to take advantage of M/R's combine() > phrase? There can be either rules embedded in in the query optimizer or hints > passed by user... > GROUP BY should benefit from this alot.. > > Any comment? > > >
