[
https://issues.apache.org/jira/browse/HIVE-11013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-11013:
------------------------------------
Attachment: HIVE-11013.patch
The problem is that removeUnionOperators gets the dummies from the target work,
and then sets them back after some machinations.
MergeJoinWork implements setDummyOps, passing dummies on to the main work, but
it doesn't implement get, so the latter goes to BaseWork and returns its own
list, which is empty.
Implementing get fixes the test.
[~vikram.dixit] [~hagleitn] you appear to have authored the code in
MergeJoinWork and GenTezUtils respectively. Can you take a look?
Do we need the java file patch for trunk?
I also left an info log when this happens.
> LLAP: MiniTez tez_join_hash test on the branch fails with NPE (initializeOp
> not called?)
> ----------------------------------------------------------------------------------------
>
> Key: HIVE-11013
> URL: https://issues.apache.org/jira/browse/HIVE-11013
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-11013.patch
>
>
> Line numbers are shifted due to logging; the NPE is at
> {noformat}
> hashMapRowGetters = new ReusableGetAdaptor[mapJoinTables.length];
> {noformat}
> So looks like mapJoinTables is null.
> I added logging to see if they could be set to null from cache, but that
> doesn't seem to be the case.
> Looks like initializeOp is not called.
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception from MapJoinOperator : null
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:428)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:872)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:87)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:872)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:643)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:656)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:659)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:755)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:315)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:278)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:271)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:257)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:361)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:339)
> ... 29 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
