[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474671#comment-16474671
]
Rohini Palaniswamy commented on PIG-5335:
-
+1
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
> Attachments: pig-5335-v01.patch, pig-5335-v02.patch
>
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472251#comment-16472251
]
Rohini Palaniswamy commented on PIG-5335:
-
The patch is setting LogicalExpression.getNextUid() instead of leaving it at
default -1 unlike other cases. Can you provide the explanation for that here
and also upload a patch with a comment giving link to your jira explanation?
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
> Attachments: pig-5335-v01.patch
>
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453493#comment-16453493
]
Koji Noguchi commented on PIG-5335:
---
bq. However, for this jira, before reaching to LogicalPlan.validate(), it is
failing within LogicalPlanBuilder's phase where ProjectStarExpander looked up
I tried moving ProjectStarExpander to later phase in LogicalPlan.validate()
but as I mentioned, some LogicalOperator depended on these projects range to be
expanded at LogicalPlanBuilder time.
So, in {{pig-5335-v01.patch}}, focused on
bq. ProjectExpression was always picking the first field (column 0) when
provided alias was not found.
and tried returning a new fieldschema with the invalid lookup fieldname. I
believe this allowed the compilation to move forward and get the right error
message inside LogicalPlan.validate()->ColumnAliasConversionVisitor.
Running tests.
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
> Attachments: pig-5335-v01.patch
>
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453484#comment-16453484
]
Koji Noguchi commented on PIG-5335:
---
Looking at why ProjectStarExpander was seeing a completely random schema with
redundant a0
{panel}
Projected field \[a0] does not exist in schema:
{color:red}a0:bytearray,a0:bytearray{color},a2:bytearray,a3:bytearray
{panel}
It was coming from LogicalPlanBuilder.buildForeachOp -->
ProjectExpression.getFieldSchema where ProjectExpression was always picking the
first field (column 0) when provided alias was not found.
This random behavior worked because later in
LogicalPlan.validate()->ColumnAliasConversionVisitor, it did correctly identify
the incorrect alias reference ("b1").
However, for this jira, before reaching to LogicalPlan.validate(), it is
failing within LogicalPlanBuilder's phase where ProjectStarExpander looked up
for range "a0..a2", and LogicalSchema.getField("a0") didn't return due to
{code:title=LogicalSchema.java}
610 StringBuilder sb = new StringBuilder("Found more than one match:
" + result.alias + ", " + fs.alias);
611 throw new FrontendException(sb.toString(), 1025);
{code}
and that exception was swallowed and got replaced with the misleading error
message shown in the description.
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453471#comment-16453471
]
Koji Noguchi commented on PIG-5335:
---
{quote}
Maybe a silly question but asking Daniel Dai. Can ProjectStarExpander and
ProjStarInUdfExpander be moved from LogicalPlanBuilder to
LogicalPlan.validate() ?
{quote}
First tried this but saw that many tests from LOCube and others started failing
since these LogicalOperator depended on having these range expanded at creation
time. I could have tried to move most of LogicalPlanBuilder steps to
LogicalPlan.validate() but that would be too much work for the minor jira with
just error message fixing.
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432406#comment-16432406
]
Koji Noguchi commented on PIG-5335:
---
bq. 2. Script itself is not using relation C and D at all. It's confusing to
see errors coming out of unused relations.
This is probably due to checks being done inside {{LogicalPlanBuilder}} when
the ForEach and others are built.
Maybe a silly question but asking [~daijy]. Can {{ProjectStarExpander}} and
{{ProjStarInUdfExpander}} be moved from LogicalPlanBuilder to
LogicalPlan.validate() ?
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PIG-5335) Error message from range projection completely misleading
[
https://issues.apache.org/jira/browse/PIG-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432356#comment-16432356
]
Koji Noguchi commented on PIG-5335:
---
Full trace.
{noformat}
2018-04-10 10:22:33,360 [main] ERROR org.apache.pig.PigServer - exception
during parsing: Error during parsing. Invalid field projection. Projected field
[a0] does not exist in schema:
a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
Failed to parse: Pig script failed to parse:
pig script failed to validate:
org.apache.pig.impl.plan.PlanValidationException: ERROR 1025: Invalid field
projection. Projected field [a0] does not exist in schema:
a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1824)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1532)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:461)
at org.apache.pig.PigServer.executeBatch(PigServer.java:486)
at org.apache.pig.PigServer.executeBatch(PigServer.java:472)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:630)
at org.apache.pig.Main.main(Main.java:175)
Caused by: pig script failed to validate:
org.apache.pig.impl.plan.PlanValidationException: ERROR 1025: Invalid field
projection. Projected field [a0] does not exist in schema:
a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
at
org.apache.pig.parser.LogicalPlanBuilder.buildForeachOp(LogicalPlanBuilder.java:1066)
at
org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:15896)
at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1933)
at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
... 11 more
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1025:
Invalid field projection. Projected field [a0] does not exist in schema:
a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
at
org.apache.pig.newplan.logical.expression.ProjectExpression.findColNum(ProjectExpression.java:191)
at
org.apache.pig.newplan.logical.expression.ProjectExpression.setColumnNumberFromAlias(ProjectExpression.java:154)
at
org.apache.pig.newplan.logical.visitor.ProjectStarExpander.visit(ProjectStarExpander.java:344)
at
org.apache.pig.parser.LogicalPlanBuilder.buildForeachOp(LogicalPlanBuilder.java:1062)
... 17 more
{noformat}
> Error message from range projection completely misleading
> -
>
> Key: PIG-5335
> URL: https://issues.apache.org/jira/browse/PIG-5335
> Project: Pig
> Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Major
>
> {code}
> A = load 'input.txt' as (a0,a1,a2,a3);
> B = FOREACH A GENERATE a0, a1, a2, a3;
> store B into '/tmp/deleteme';
> C = FOREACH A GENERATE a0, b1, a2, a3;
> D = FOREACH C GENERATE a0..a2;
> (end of script, no store, nothing)
> {code}
> Error message
> {panel}
> 2018-04-10 10:22:33,360 \[main] ERROR org.apache.pig.PigServer - exception
> during parsing: Error during parsing. Invalid field projection. Projected
> field \[a0] does not exist in schema:
> a0:bytearray,a0:bytearray,a2:bytearray,a3:bytearray.
> {panel}
> At least two issues.
> # Error should be about FOREACH for C referencing non-existing field 'b1'.
> But the error message is saying something about 'a0'.
> # Script itself is not using relation C and D at all. It's confusing to see
> errors coming out of unused relations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
