pig gets confused about schema, when joining a table that has a known schema
with one that doesn't
--------------------------------------------------------------------------------------------------
Key: PIG-542
URL: https://issues.apache.org/jira/browse/PIG-542
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: types_branch
Environment: types branch, running in local mode
Reporter: Christopher Olston
query:
A = load '/data/A' using myLoadFunc('...');
A1 = foreach (group A by ($8)) generate group, COUNT($1);
B = load '/data/B';
J = join A1 by $0, B by $0;
J1 = foreach J generate $0, $1, $3; <----- crashes on attempt to parse
this line.
problem:
It knows the schema of A1 but not of B -- but it seems to think B has only
one field.
error message (on parsing J1=... line):
Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Out of
bound access. Trying to access non-existent column: 3. Schema {ID10::group:
bytearray,long,bytearray} has 3 column(s).
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.DollarVar(QueryParser.ja
va:5764)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.ColOrSpec(QueryParser.ja
va:5713)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser
.java:4018)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.ja
va:3915)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.jav
a:3869)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(Query
Parser.java:3778)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser
.java:3704)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.ja
va:3670)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItem(Qu
eryParser.java:3596)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItemLis
t(QueryParser.java:3519)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.GenerateStatement(QueryP
arser.java:3463)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.
java:2939)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParse
r.java:2342)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.jav
a:979)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:75
5)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:5
50)
at
org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder
.java:60)
at org.apache.pig.PigServer.parseQuery(PigServer.java:295)
... 16 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
