New logical plan: logical plan get confused if we generate the same field twice
in ForEach
------------------------------------------------------------------------------------------
Key: PIG-1732
URL: https://issues.apache.org/jira/browse/PIG-1732
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
The following script fail:
{code}
a = load '1.txt' as (a0, a1);
b = load '2.txt' as (b0, b1, b2);
c = foreach a generate a0, a1, a1 as a2;
d = union b, c;
e = foreach d generate $1;
explain e;
{code}
Error message:
ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to
explain alias e
at org.apache.pig.PigServer.explain(PigServer.java:958)
at
org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:353)
at
org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:285)
at
org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:248)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:605)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:327)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:498)
at org.apache.pig.Main.main(Main.java:107)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2042:
Error in new logical plan. Try -Dpig.usenewlogicalplan=false.
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:309)
at org.apache.pig.PigServer.compilePp(PigServer.java:1354)
at org.apache.pig.PigServer.explain(PigServer.java:927)
... 10 more
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000:
Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune
at
org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:120)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:277)
... 12 more
Caused by: java.lang.NullPointerException
at
org.apache.pig.newplan.logical.relational.LOUnion.getSchema(LOUnion.java:75)
at
org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:127)
at
org.apache.pig.newplan.logical.relational.LOUnion.accept(LOUnion.java:102)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at
org.apache.pig.newplan.logical.optimizer.SchemaPatcher.transformed(SchemaPatcher.java:43)
at
org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:112)
The problem is caused by c = foreach a generate a0, a1, a1 as a2, in which we
will generate two fields share one uid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
