[jira] [Commented] (HIVE-9886) Hive on tez: NPE when converting join to SMB in sub-query

2015-03-19 Thread Gunther Hagleitner (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369946#comment-14369946
 ] 

Gunther Hagleitner commented on HIVE-9886:
--

[~vikram.dixit] can you reupload? to see if we can get a clean run.

 Hive on tez: NPE when converting join to SMB in sub-query
 -

 Key: HIVE-9886
 URL: https://issues.apache.org/jira/browse/HIVE-9886
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 1.0.0, 1.1.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Priority: Critical
 Attachments: HIVE-9886.1.patch, HIVE-9886.2.patch, HIVE-9886.3.patch, 
 HIVE-9886.4.patch, HIVE-9886.5.patch, HIVE-9886.6.patch


 {code}
 set hive.auto.convert.sortmerge.join = true;
 create table t1(
 id string,
 od string);
 create table t2(
 id string,
 od string);
 select vt1.id from
 (select rt1.id from
 (select t1.id, row_number() over (partition by id order by od desc) as row_no 
 from t1) rt1
 where rt1.row_no=1) vt1
 join
 (select rt2.id from
 (select t2.id, row_number() over (partition by id order by od desc) as row_no 
 from t2) rt2
 where rt2.row_no=1) vt2
 where vt1.id=vt2.id;
 {code}
 throws NPE:
 {code}
   at 
 org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:146)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
   at 
 org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.getValueObjectInspectors(AbstractMapJoinOperator.java:96)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getJoinOutputObjectInspector(CommonJoinOperator.java:167)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:310)
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:72)
   at 
 org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.initializeOp(CommonMergeJoinOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:66)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:410)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.initializeOp(PTFOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at

[jira] [Commented] (HIVE-9886) Hive on tez: NPE when converting join to SMB in sub-query

2015-03-07 Thread Vikram Dixit K (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351829#comment-14351829
 ] 

Vikram Dixit K commented on HIVE-9886:
--

Looks like test results are not getting posted:

http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2973/testReport/

 Hive on tez: NPE when converting join to SMB in sub-query
 -

 Key: HIVE-9886
 URL: https://issues.apache.org/jira/browse/HIVE-9886
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 1.0.0, 1.1.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Priority: Critical
 Attachments: HIVE-9886.1.patch, HIVE-9886.2.patch, HIVE-9886.3.patch, 
 HIVE-9886.4.patch, HIVE-9886.5.patch, HIVE-9886.6.patch


 {code}
 set hive.auto.convert.sortmerge.join = true;
 create table t1(
 id string,
 od string);
 create table t2(
 id string,
 od string);
 select vt1.id from
 (select rt1.id from
 (select t1.id, row_number() over (partition by id order by od desc) as row_no 
 from t1) rt1
 where rt1.row_no=1) vt1
 join
 (select rt2.id from
 (select t2.id, row_number() over (partition by id order by od desc) as row_no 
 from t2) rt2
 where rt2.row_no=1) vt2
 where vt1.id=vt2.id;
 {code}
 throws NPE:
 {code}
   at 
 org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:146)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
   at 
 org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.getValueObjectInspectors(AbstractMapJoinOperator.java:96)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getJoinOutputObjectInspector(CommonJoinOperator.java:167)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:310)
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:72)
   at 
 org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.initializeOp(CommonMergeJoinOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:66)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:410)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.initializeOp(PTFOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at

[jira] [Commented] (HIVE-9886) Hive on tez: NPE when converting join to SMB in sub-query

2015-03-06 Thread Gunther Hagleitner (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351251#comment-14351251
 ] 

Gunther Hagleitner commented on HIVE-9886:
--

+1

 Hive on tez: NPE when converting join to SMB in sub-query
 -

 Key: HIVE-9886
 URL: https://issues.apache.org/jira/browse/HIVE-9886
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 1.0.0, 1.1.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Priority: Critical
 Attachments: HIVE-9886.1.patch, HIVE-9886.2.patch, HIVE-9886.3.patch, 
 HIVE-9886.4.patch, HIVE-9886.5.patch


 {code}
 set hive.auto.convert.sortmerge.join = true;
 create table t1(
 id string,
 od string);
 create table t2(
 id string,
 od string);
 select vt1.id from
 (select rt1.id from
 (select t1.id, row_number() over (partition by id order by od desc) as row_no 
 from t1) rt1
 where rt1.row_no=1) vt1
 join
 (select rt2.id from
 (select t2.id, row_number() over (partition by id order by od desc) as row_no 
 from t2) rt2
 where rt2.row_no=1) vt2
 where vt1.id=vt2.id;
 {code}
 throws NPE:
 {code}
   at 
 org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:146)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
   at 
 org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.getValueObjectInspectors(AbstractMapJoinOperator.java:96)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getJoinOutputObjectInspector(CommonJoinOperator.java:167)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:310)
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:72)
   at 
 org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.initializeOp(CommonMergeJoinOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:66)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:410)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.initializeOp(PTFOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at 
 org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:116)
   ... 14 more
 {code}



--
This message was sent by

[jira] [Commented] (HIVE-9886) Hive on tez: NPE when converting join to SMB in sub-query

2015-03-06 Thread Gunther Hagleitner (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351190#comment-14351190
 ] 

Gunther Hagleitner commented on HIVE-9886:
--

I think we should have some follow up jiras:

- Trait set propagation for map join (one side sort column propagation)
- ReduceRecordProc should be able to handle smb
- Think through the assumption that multiple smbs in the same vertex are always 
merged (smb-mj-smb-mj)

For now the safest might be to set the reducesink count once and not update 
while transforming the plan. That will allow smb followed by any number of mj 
in the first (map) vertex, but much nothing else (as far as smb is concerned).

 Hive on tez: NPE when converting join to SMB in sub-query
 -

 Key: HIVE-9886
 URL: https://issues.apache.org/jira/browse/HIVE-9886
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 1.0.0, 1.1.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Priority: Critical
 Attachments: HIVE-9886.1.patch, HIVE-9886.2.patch


 {code}
 set hive.auto.convert.sortmerge.join = true;
 create table t1(
 id string,
 od string);
 create table t2(
 id string,
 od string);
 select vt1.id from
 (select rt1.id from
 (select t1.id, row_number() over (partition by id order by od desc) as row_no 
 from t1) rt1
 where rt1.row_no=1) vt1
 join
 (select rt2.id from
 (select t2.id, row_number() over (partition by id order by od desc) as row_no 
 from t2) rt2
 where rt2.row_no=1) vt2
 where vt1.id=vt2.id;
 {code}
 throws NPE:
 {code}
   at 
 org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:146)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
   at 
 org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.getValueObjectInspectors(AbstractMapJoinOperator.java:96)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getJoinOutputObjectInspector(CommonJoinOperator.java:167)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:310)
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:72)
   at 
 org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.initializeOp(CommonMergeJoinOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:66)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:410)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.initializeOp(PTFOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at

[jira] [Commented] (HIVE-9886) Hive on tez: NPE when converting join to SMB in sub-query

2015-03-06 Thread Gunther Hagleitner (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350969#comment-14350969
 ] 

Gunther Hagleitner commented on HIVE-9886:
--

There's a lot of ws changes in the patch that make it hard to review. I'm 
usually not against fixing identation etc, but in this case it seems to make 
the indentation worse in a lot of places.

 Hive on tez: NPE when converting join to SMB in sub-query
 -

 Key: HIVE-9886
 URL: https://issues.apache.org/jira/browse/HIVE-9886
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 1.0.0, 1.1.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Priority: Critical
 Attachments: HIVE-9886.1.patch


 {code}
 set hive.auto.convert.sortmerge.join = true;
 create table t1(
 id string,
 od string);
 create table t2(
 id string,
 od string);
 select vt1.id from
 (select rt1.id from
 (select t1.id, row_number() over (partition by id order by od desc) as row_no 
 from t1) rt1
 where rt1.row_no=1) vt1
 join
 (select rt2.id from
 (select t2.id, row_number() over (partition by id order by od desc) as row_no 
 from t2) rt2
 where rt2.row_no=1) vt2
 where vt1.id=vt2.id;
 {code}
 throws NPE:
 {code}
   at 
 org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:146)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:162)
   at 
 org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
   at 
 org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
   at 
 org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.getValueObjectInspectors(AbstractMapJoinOperator.java:96)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getJoinOutputObjectInspector(CommonJoinOperator.java:167)
   at 
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:310)
   at 
 org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:72)
   at 
 org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.initializeOp(CommonMergeJoinOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:66)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:410)
   at 
 org.apache.hadoop.hive.ql.exec.PTFOperator.initializeOp(PTFOperator.java:89)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
   at 
 org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
   at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
   at