[
https://issues.apache.org/jira/browse/PIG-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13902256#comment-13902256
]
Mona Chitnis commented on PIG-3000:
-----------------------------------
Patch updated to RB.
Patch updated to handle the "Projection with nothing to reference issue" which
was coming from the innerLoad of the altered ForEach.
Doing an explain on new plan gives correct new optimized plan. The commented
out part in the patch is because I observed that this was getting automatically
done by SchemaPatcher and ProjectionPatcher listeners in the
LogicalPlanOptimizer. However, this gives variable results for the uids and
following error -
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2229:
Couldn't find matching uid -1 for project
org.apache.pig.builtin.upper_17:(Name: Project Type: chararray Uid: 38 Input: 0
Column: 0)
at
org.apache.pig.newplan.logical.optimizer.ProjectionPatcher$ProjectionRewriter.visit(ProjectionPatcher.java:91)
at
org.apache.pig.newplan.logical.expression.ProjectExpression.accept(ProjectExpression.java:215)
(Where upper_17 is an example unique alias generated for the UserFuncExpression
operator in new plan)
Any help is appreciated. This patch excludes unit tests and will upload all in
next patch after fixing this issue.
> Optimize nested foreach
> -----------------------
>
> Key: PIG-3000
> URL: https://issues.apache.org/jira/browse/PIG-3000
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.10.0
> Reporter: Richard Ding
> Assignee: Mona Chitnis
>
> In this Pig script:
> {code}
> A = load 'data' as (a:chararray);
> B = foreach A { c = UPPER(a); generate ((c eq 'TEST') ? 1 : 0), ((c eq 'DEV')
> ? 1 : 0); }
> {code}
> The Eval function UPPER is called twice for each record.
> This should be optimized so that the UPPER is called only once for each record
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
