[ 
https://issues.apache.org/jira/browse/PIG-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659144#action_12659144
 ] 

Santhosh Srinivasan commented on PIG-577:
-----------------------------------------

The correct statement to make mimic the outer join semantics will be:

{code}
D = FOREACH C GENERATE group, flatten((not IsEmpty(A) ? A : 
(bag{tuple(chararray, int, float)}){(null, null, null)})), flatten((not 
IsEmpty(B) ? B : (bag{tuple(chararray, int, chararray, 
float)}){(null,null,null, null)}));
{code}

However, this exposed a bag in the type checker where the schemas of the LHS 
and RHS do not match. The bag with the null constants has a tuple and relation 
A(or B) has a schema without the tuple. This issue was resolved in PIG-449. The 
solution proposed in PIG-449 has to be extended to schema comparisons that 
involve bags.

{code}
2008-12-24 10:31:58,529 [main] ERROR org.apache.pig.tools.grunt.Grunt - Two 
inputs of BinCond must have compatible schemas

2008-12-24 10:31:58,529 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
org.apache.pig.impl.logicalLayer.FrontendException: Unable to describe schema 
for alias D
        at org.apache.pig.PigServer.dumpSchema(PigServer.java:367)
        at 
org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:153)
        at 
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:188)
        at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:71)
        at org.apache.pig.Main.main(Main.java:302)
Caused by: org.apache.pig.impl.plan.PlanValidationException: An unexpected 
exception caused the validation to stop
        at 
org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
        at 
org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:79)
        at org.apache.pig.PigServer.compileLp(PigServer.java:687)
        at org.apache.pig.PigServer.dumpSchema(PigServer.java:360)
        ... 5 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
Cannot resolve ForEach output schema.
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2731)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:122)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:41)
        at 
org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at 
org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
        ... 10 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
Problem during evaluaton of BinCond output type
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:1913)
        at org.apache.pig.impl.logicalLayer.LOBinCond.visit(LOBinCond.java:88)
        at org.apache.pig.impl.logicalLayer.LOBinCond.visit(LOBinCond.java:27)
        at 
org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.checkInnerPlan(TypeCheckingVisitor.java:2812)
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2720)
        ... 15 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
Two inputs of BinCond must have compatible schemas
        at 
org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:1903)
        ... 21 more

{code}

> outer join query looses name information
> ----------------------------------------
>
>                 Key: PIG-577
>                 URL: https://issues.apache.org/jira/browse/PIG-577
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>             Fix For: types_branch
>
>
> The following query:
> A = LOAD 'student_data' AS (name: chararray, age: int, gpa: float);
> B = LOAD 'voter_data' AS (name: chararray, age: int, registration: chararray, 
> contributions: float);
> C = COGROUP A BY name, B BY name;
> D = FOREACH C GENERATE group, flatten((IsEmpty(A) ? null : A)), 
> flatten((IsEmpty(B) ? null : B));
> describe D;
> E = FOREACH D GENERATE A::gpa, B::contributions;
> Give the following error: (Even though describe shows correct information: D: 
> {group: chararray,A::name: chararray,A::age: int,A::gpa: float,B::name: 
> chararray,B::age: int,B::registration: chararray,B::contributions: float}
> java.io.IOException: Invalid alias: A::gpa in {group: 
> chararray,bytearray,bytearray}
>         at org.apache.pig.PigServer.parseQuery(PigServer.java:298)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:263)
>         at 
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:439)
>         at 
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:249)
>         at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
>         at org.apache.pig.Main.main(Main.java:306)
> Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Invalid 
> alias: A::gpa in {group: chararray,bytearray,bytearray}
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AliasFieldOrSpec(QueryParser.java:5930)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.ColOrSpec(QueryParser.java:5788)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:3974)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:3871)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:3825)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:3734)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:3660)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:3626)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItem(QueryParser.java:3552)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItemList(QueryParser.java:3462)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.GenerateStatement(QueryParser.java:3419)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:2894)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:2309)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:966)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:742)
>         at 
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:537)
>         at 
> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:60)
>         at org.apache.pig.PigServer.parseQuery(PigServer.java:295)
>         ... 6 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to