Sergey Shelukhin created HIVE-15278:
---------------------------------------
Summary: PTF+SortMergeJoin = NPE
Key: HIVE-15278
URL: https://issues.apache.org/jira/browse/HIVE-15278
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Manifests as
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.persistence.PTFRowContainer.first(PTFRowContainer.java:115)
at
org.apache.hadoop.hive.ql.exec.PTFPartition.iterator(PTFPartition.java:114)
at
org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:340)
at
org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:114)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
... 29 more
{noformat}
It's actually a somewhat subtle ordering problem in sortmerge - as it stands,
it calls different branches of the tree in closeOp after they themselves have
already been closed. Other operators that clean stuff up in close may result in
different errors. The common pattern is
{noformat}
1125 at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
1126 at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
1127 at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:404)
...
1131 at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:428)
1132 at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:388)
1133 at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
...
1139 at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:294)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
