[
https://issues.apache.org/jira/browse/PIG-479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair resolved PIG-479.
-------------------------------
Resolution: Won't Fix
Quoting Alan's comment in PIG-750 -
"Our performance tests have shown that having combiner and non-combiner
functions in the same MR job actually severly slows things down. We suspect
that this is because you have to pass the bags for the non-combiner functions
through the combiner and you pay for the multiple (de)serialization passes. "
Marking this jira as 'Won't Fix' .
I have created PIG-1735 to address (lack of) use of combiner in case of queries
co-group , such as the one in Scott's example above.
> PERFORMANCE: more extensive use of the combier
> ----------------------------------------------
>
> Key: PIG-479
> URL: https://issues.apache.org/jira/browse/PIG-479
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.2.0
> Reporter: Olga Natkovich
> Assignee: Thejas M Nair
>
> On types branch, the combiner is used anytime a foreach includes only simple
> projections and/or algebraic functions. It would also be useful to invoke
> the combiner in cases where algebraic and non-algebraic operations are mixed,
> or where expression evaluation is included in the foreach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
