NPE in reducer when using cross join with order by in sub select.
-----------------------------------------------------------------
Key: HIVE-2345
URL: https://issues.apache.org/jira/browse/HIVE-2345
Project: Hive
Issue Type: Bug
Affects Versions: 0.7.1
Reporter: Inbal (Roza) Rosenshtock
When running the following query:
select * from
(select * from temp_table order by (id)) t
Join
(select * from temp_table) b
We get the exceptions below:
In stage 1:
2011-08-03 23:54:33,214 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:431)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:144)
... 14 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:129)
... 14 more
2011-08-03 23:54:33,217 INFO org.apache.hadoop.mapred.Task: Runnning cleanup
for the task
In stage 2:
2011-08-03 23:55:18,905 INFO org.apache.hadoop.hive.ql.exec.ExtractOperator:
Initialization Done 3 OP
2011-08-03 23:55:18,908 FATAL ExecReducer: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.startGroup(CommonJoinOperator.java:341)
at org.apache.hadoop.hive.ql.exec.Operator.startGroup(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.Operator.startGroup(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:213)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:468)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
2011-08-03 23:55:18,911 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2011-08-03 23:55:18,913 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:268)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:468)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.startGroup(CommonJoinOperator.java:341)
at org.apache.hadoop.hive.ql.exec.Operator.startGroup(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.Operator.startGroup(Operator.java:489)
at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:213)
... 7 more
2011-08-03 23:55:18,916 INFO org.apache.hadoop.mapred.Task: Runnning cleanup
for the task
The query succeeds when:
We add the mapjoin :
select /*+ MAPJOIN(b) */ * from
(select * from temp_table order by (id)) t
Join
(select * from temp_table) b
or
select /*+ MAPJOIN(t) */ * from
(select * from temp_table order by (id)) t
Join
(select * from temp_table) b
Or use equi join:
select * from
(select * from temp_table order by (id)) t
Join
(select * from temp_table) b
on (b.id=t.id)
Note: The size of the table temp_table doesn't matter.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
