[
https://issues.apache.org/jira/browse/PIG-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-1568:
----------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
test-patch result:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 6 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
Patch committed. Thanks Xuefu!
> Optimization rule FilterAboveForeach is too restrictive and doesn't handle
> project * correctly
> ----------------------------------------------------------------------------------------------
>
> Key: PIG-1568
> URL: https://issues.apache.org/jira/browse/PIG-1568
> Project: Pig
> Issue Type: Bug
> Reporter: Xuefu Zhang
> Assignee: Xuefu Zhang
> Fix For: 0.8.0
>
> Attachments: jira-1568-1.patch, jira-1568-1.patch
>
>
> FilterAboveForeach rule is to optimize the plan by pushing up filter above
> previous foreach operator. However, during code review, two major problems
> were found:
> 1. Current implementation assumes that if no projection is found in the
> filter condition then all columns from foreach are projected. This issue
> prevents the following optimization:
> A = LOAD 'file.txt' AS (a(u,v), b, c);
> B = FOREACH A GENERATE $0, b;
> C = FILTER B BY 8 > 5;
> STORE C INTO 'empty';
> 2. Current implementation doesn't handle * probjection, which means project
> all columns. As a result, it wasn't able to optimize the following:
> A = LOAD 'file.txt' AS (a(u,v), b, c);
> B = FOREACH A GENERATE $0, b;
> C = FILTER B BY Identity.class.getName(*) > 5;
> STORE C INTO 'empty';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
