[
https://issues.apache.org/jira/browse/PIG-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176391#comment-16176391
]
Koji Noguchi commented on PIG-5271:
-----------------------------------
bq. Koji Noguchi I think you forgot to commit TEZC-Union-22.gld from this
patch! Could you please commit this file too?
Ouch. It was from my own patch and I still missed adding this file. Thanks
for pointing it out. Added now.
> StackOverflowError when compiling in Tez mode (with union and replicated join)
> ------------------------------------------------------------------------------
>
> Key: PIG-5271
> URL: https://issues.apache.org/jira/browse/PIG-5271
> Project: Pig
> Issue Type: Bug
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Fix For: 0.18.0
>
> Attachments: pig-5271-v01.patch, pig-5271-v02.patch,
> pig-5271-v03.patch
>
>
> Sample script
> {code}
> a4 = LOAD 'studentnulltab10k' as (name, age:int, gpa:float);
> a4_1 = filter a4 by gpa is null or gpa >= 3.9;
> a4_2 = filter a4 by gpa < 1;
> b4 = union a4_1, a4_2;
> b4_1 = filter b4 by age < 30;
> b4_2 = foreach b4 generate name, age, FLOOR(gpa) as gpa;
> c4 = load 'voternulltab10k' as (name, age, registration, contributions);
> d4 = join b4_2 by name, c4 by name using 'replicated';
> e4 = foreach d4 generate b4_2::name as name, b4_2::age as age, gpa,
> registration, contributions;
> f4 = order e4 by name, age DESC;
> store f4 into 'tmp_table_4' ;
> a5_1 = filter a4 by gpa is null or gpa <= 3.9;
> a5_2 = filter a4 by gpa < 2;
> b5 = union a5_1, a5_2;
> d5 = join c4 by name, b5 by name using 'replicated';
> store d5 into 'tmp_table_5' ;
> {code}
> This script fails to compile with StackOverflowError.
> {noformat}
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
> Pig Stack Trace
> ---------------
> ERROR 2998: Unhandled internal error. null
> java.lang.StackOverflowError
> at java.lang.reflect.Constructor.newInstance(Constructor.java:415)
> at java.lang.Class.newInstance(Class.java:442)
> at org.apache.pig.impl.util.Utils.mergeCollection(Utils.java:490)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.doAllPredecessors(DependencyOrderWalker.java:101)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.doAllPredecessors(DependencyOrderWalker.java:105)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.doAllPredecessors(DependencyOrderWalker.java:105)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.doAllPredecessors(DependencyOrderWalker.java:105)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.doAllPredecessors(DependencyOrderWalker.java:105)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.doAllPredecessors(DependencyOrderWalker.java:105)
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
