[
https://issues.apache.org/jira/browse/TAJO-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364763#comment-14364763
]
ASF GitHub Bot commented on TAJO-1368:
--------------------------------------
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/402#issuecomment-82199826
Hi @ykrips thanks for your nice patch.
It looks good, but I have another suggestion even though it is not directly
related with this issue.
I think that ```union``` queries are popularly used for ETLs. So, I tested
the following query. (It is a CTAS statement of the case 1 of TAJO-1368.)
```
create table test using parquet as
select * from
(select c_custkey, c_nationkey from customer where c_nationkey < 0
union all
select c_custkey, c_nationkey from customer where c_nationkey > 0
) a
union all
select * from
(select c_custkey, c_nationkey from customer where c_nationkey < 0
union all
select c_custkey, c_nationkey from customer where c_nationkey > 0
) b
```
And, I found the following error.
```
2015-03-17 17:29:42,257 ERROR: org.apache.tajo.querymaster.QueryMasterTask
(startQuery(368)) -
java.lang.NullPointerException
at
org.apache.tajo.engine.planner.global.GlobalPlanner.buildNoPartitionedStorePlan(GlobalPlanner.java:1113)
at
org.apache.tajo.engine.planner.global.GlobalPlanner.buildStorePlan(GlobalPlanner.java:1037)
at
org.apache.tajo.engine.planner.global.GlobalPlanner.access$600(GlobalPlanner.java:68)
at
org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitCreateTable(GlobalPlanner.java:1570)
at
org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitCreateTable(GlobalPlanner.java:1141)
at
org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:126)
at
org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:151)
at
org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitRoot(GlobalPlanner.java:1146)
at
org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitRoot(GlobalPlanner.java:1141)
at
org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:60)
at
org.apache.tajo.engine.planner.global.GlobalPlanner.build(GlobalPlanner.java:147)
at
org.apache.tajo.querymaster.QueryMasterTask.startQuery(QueryMasterTask.java:360)
at
org.apache.tajo.querymaster.QueryMasterTask.start(QueryMasterTask.java:175)
at
org.apache.tajo.querymaster.QueryMaster$QueryStartEventHandler.handle(QueryMaster.java:416)
at
org.apache.tajo.querymaster.QueryMaster$QueryStartEventHandler.handle(QueryMaster.java:403)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
at java.lang.Thread.run(Thread.java:745)
```
This error seems to be caused by misfunctioning of
```GlobalPlanner.hasUnionChild()```.
I think it would be great if you fix this bug, too.
> Exceptions during processing nested union queries
> -------------------------------------------------
>
> Key: TAJO-1368
> URL: https://issues.apache.org/jira/browse/TAJO-1368
> Project: Tajo
> Issue Type: Bug
> Components: distributed query plan, worker
> Reporter: Jihoon Son
> Assignee: Jihun Kang
>
> This error is reported from the Apache Tajo Korea user group
> (https://groups.google.com/forum/#!topic/tajo-user-kr/tipQj6cv-k0).
> You can reproduce the reported errors as follows:
> {noformat}
> default> create table test_union_all (cd int, nm text);
> default> insert into test_union_all select 1,'aaa';
> default> insert into test_union_all select 1,'bbb';
> {noformat}
> *Case 1*
> Distributed query planner emits NPE as follows:
> {noformat}
> default> select * from
> (
> select * from test_union_all
> union all
> select * from test_union_all
> )a
> union all
> select * from
> (
> select * from test_union_all
> union all
> select * from test_union_all
> )a;
> 2015-03-03 13:59:12,738 ERROR org.apache.tajo.querymaster.QueryMasterTask:
> java.lang.NullPointerException
> at
> org.apache.tajo.engine.planner.global.DataChannel.<init>(DataChannel.java:57)
> at
> org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitUnion(GlobalPlanner.java:1405)
> at
> org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitUnion(GlobalPlanner.java:1140)
> at
> org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:96)
> at
> org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:151)
> at
> org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitRoot(GlobalPlanner.java:1145)
> at
> org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitRoot(GlobalPlanner.java:1140)
> at
> org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:60)
> at
> org.apache.tajo.engine.planner.global.GlobalPlanner.build(GlobalPlanner.java:146)
> at
> org.apache.tajo.querymaster.QueryMasterTask.startQuery(QueryMasterTask.java:357)
> at
> org.apache.tajo.querymaster.QueryMasterTask.start(QueryMasterTask.java:172)
> at
> org.apache.tajo.querymaster.QueryMaster$QueryStartEventHandler.handle(QueryMaster.java:410)
> at
> org.apache.tajo.querymaster.QueryMaster$QueryStartEventHandler.handle(QueryMaster.java:397)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> *Case 2*
> TaskRunner emits NPE as follows:
> {noformat}
> default> select * from (
> select cd, nm
> from
> (
> select cd, nm
> from
> (
> select cd, nm
> from test_union_all
> )a
> union all
> select cd, nm
> from
> (
> select cd, nm
> from test_union_all
> )a
> ) a
> )a;
> 2015-03-03 14:05:02,960 ERROR org.apache.tajo.worker.Task:
> java.lang.IllegalArgumentException: URI scheme is not "file"
> java.io.IOException: java.lang.IllegalArgumentException: URI scheme is not
> "file"
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.init(RawFile.java:485)
> at
> org.apache.tajo.engine.planner.physical.StoreTableExec.openNewFile(StoreTableExec.java:110)
> at
> org.apache.tajo.engine.planner.physical.StoreTableExec.init(StoreTableExec.java:79)
> at org.apache.tajo.worker.Task.run(Task.java:405)
> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:276)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: URI scheme is not "file"
> at java.io.File.<init>(File.java:421)
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.init(RawFile.java:480)
> ... 5 more
> 2015-03-03 14:05:02,963 ERROR org.apache.tajo.worker.TaskRunner:
> java.lang.NullPointerException
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.close(RawFile.java:756)
> at
> org.apache.tajo.engine.planner.physical.StoreTableExec.close(StoreTableExec.java:143)
> at org.apache.tajo.worker.Task.run(Task.java:417)
> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:276)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
