[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15873879#comment-15873879 ] Hive QA commented on HIVE-15904: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12853498/HIVE-15904.6.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10246 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic] (batchId=43) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2] (batchId=152) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=223) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] (batchId=133) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=211) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3652/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3652/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3652/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12853498 - PreCommit-HIVE-Build > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, HIVE-15904.5.patch, > HIVE-15904.6.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > or
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15873857#comment-15873857 ] Hive QA commented on HIVE-15904: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12853487/HIVE-15904.5.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10246 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_semijoin_reduction_2] (batchId=70) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic] (batchId=43) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2] (batchId=152) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=223) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] (batchId=133) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3651/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3651/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3651/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12853487 - PreCommit-HIVE-Build > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, HIVE-15904.5.patch, table_18.q, > table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apac
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872759#comment-15872759 ] Jason Dere commented on HIVE-15904: --- dynamic_semijoin_reduction_2 needs to be added to testconfiguration.properties > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.j
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872750#comment-15872750 ] Hive QA commented on HIVE-15904: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12853361/HIVE-15904.4.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10246 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_semijoin_reduction_2] (batchId=70) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2] (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] (batchId=133) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=211) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3634/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3634/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3634/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12853361 - PreCommit-HIVE-Build > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.serv
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872747#comment-15872747 ] Jason Dere commented on HIVE-15904: --- Yeah, i took a couple of stabs at trying to trim simplify and failed. Will give it one more try. > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.w
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872735#comment-15872735 ] Gunther Hagleitner commented on HIVE-15904: --- +1. Nit: I like that it has a test, but it would have been nice to trim the test down a bit. Hard to follow. > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872654#comment-15872654 ] Jason Dere commented on HIVE-15904: --- [~hagleitn] can you review? > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.java:74) > at > org.apache.hadoop.hive.ql.
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868300#comment-15868300 ] Jason Dere commented on HIVE-15904: --- Couple comments: 1) Can you move the test to a new .q file, to avoid cluttering the old test 2) If we bail out of the loop using the new check you added (exprNodeDesc.getChildren() != null), then exprNodeDesc is not a ExprNodeColDesc .. in which case I am not sure the subsequent logic handles this case correctly. Can you check if your testcase is causing NPE without your patch? > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Deepak Jaiswal > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867193#comment-15867193 ] Deepak Jaiswal commented on HIVE-15904: --- Apache rb link, https://reviews.apache.org/r/56695/ > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Deepak Jaiswal > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.java:74) > at > org.apach
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867086#comment-15867086 ] Hive QA commented on HIVE-15904: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852669/HIVE-15904.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10238 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2] (batchId=152) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] (batchId=133) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3552/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3552/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3552/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12852669 - PreCommit-HIVE-Build > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Deepak Jaiswal > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getR
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866721#comment-15866721 ] Hive QA commented on HIVE-15904: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852643/HIVE-15904.2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3547/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3547/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3547/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-02-14 21:28:11.281 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3547/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-02-14 21:28:11.284 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 8432dd8 HIVE-15901 : LLAP: incorrect usage of gap cache (Sergey Shelukhin, reviewed by Prasanth Jayachandran) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 8432dd8 HIVE-15901 : LLAP: incorrect usage of gap cache (Sergey Shelukhin, reviewed by Prasanth Jayachandran) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-02-14 21:28:12.187 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java: No such file or directory The patch does not appear to apply with p0, p1, or p2 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12852643 - PreCommit-HIVE-Build > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Deepak Jaiswal > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, table_18.q, > table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal19
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866720#comment-15866720 ] Hive QA commented on HIVE-15904: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852643/HIVE-15904.2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3546/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3546/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3546/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-02-14 21:27:28.878 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-3546/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-02-14 21:27:28.881 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 8432dd8 HIVE-15901 : LLAP: incorrect usage of gap cache (Sergey Shelukhin, reviewed by Prasanth Jayachandran) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 8432dd8 HIVE-15901 : LLAP: incorrect usage of gap cache (Sergey Shelukhin, reviewed by Prasanth Jayachandran) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-02-14 21:27:29.962 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java: No such file or directory The patch does not appear to apply with p0, p1, or p2 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12852643 - PreCommit-HIVE-Build > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Deepak Jaiswal > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, table_18.q, > table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal19