[
https://issues.apache.org/jira/browse/PIG-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894388#action_12894388
]
Mridul Muralidharan commented on PIG-1530:
------------------------------------------
Cant edit comments .. to add to previous comment :
To clarify my prev comment, a general pattern would not stop just at co-group,
but at most (if not all) operators which dont modify the relevant fields.
> PIG Logical Optimization: Push LOFilter above LOCogroup
> --------------------------------------------------------
>
> Key: PIG-1530
> URL: https://issues.apache.org/jira/browse/PIG-1530
> Project: Pig
> Issue Type: New Feature
> Components: impl
> Reporter: Swati Jain
> Assignee: Swati Jain
> Priority: Minor
> Fix For: 0.8.0
>
>
> Consider the following:
> {noformat}
> A = load '<any file>' USING PigStorage(',') as (a1:int,a2:int,a3:int);
> B = load '<any file>' USING PigStorage(',') as (b1:int,b2:int,b3:int);
> G = COGROUP A by (a1,a2) , B by (b1,b2);
> D = Filter G by group.$0 + 5 > group.$1;
> explain D;
> {noformat}
> In the above example, LOFilter can be pushed above LOCogroup. Note there are
> some tricky NULL issues to think about when the Cogroup is not of type INNER
> (Similar to issues that need to be thought through when pushing LOFilter on
> the right side of a LeftOuterJoin).
> Also note that typically the LOFilter in user programs will be below a
> ForEach-Cogroup pair. To make this really useful, we need to also implement
> LOFilter pushed across ForEach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
