[ https://issues.apache.org/jira/browse/HIVE-21778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887564#comment-16887564 ]
Hive QA commented on HIVE-21778: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12975098/HIVE-21778.1.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 16674 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original] (batchId=183) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=178) org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query93] (batchId=297) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query93] (batchId=295) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query93] (batchId=295) org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeaderEmptyConfig.testHouseKeepingThreadExistence (batchId=241) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/18074/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18074/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18074/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12975098 - PreCommit-HIVE-Build > CBO: "Struct is not null" gets evaluated as `nullable` always causing filter > miss in the query > ---------------------------------------------------------------------------------------------- > > Key: HIVE-21778 > URL: https://issues.apache.org/jira/browse/HIVE-21778 > Project: Hive > Issue Type: Bug > Components: CBO > Affects Versions: 4.0.0, 2.3.5 > Reporter: Rajesh Balamohan > Assignee: Vineet Garg > Priority: Major > Attachments: HIVE-21778.1.patch, test_null.q, test_null.q.out > > > {noformat} > drop table if exists test_struct; > CREATE external TABLE test_struct > ( > f1 string, > demo_struct struct<f1:string, f2:string, f3:string>, > datestr string > ); > set hive.cbo.enable=true; > explain select * from etltmp.test_struct where datestr='2019-01-01' and > demo_struct is not null; > STAGE PLANS: > Stage: Stage-0 > Fetch Operator > limit: -1 > Processor Tree: > TableScan > alias: test_struct > filterExpr: (datestr = '2019-01-01') (type: boolean) <----- Note > that demo_struct filter is not added here > Filter Operator > predicate: (datestr = '2019-01-01') (type: boolean) > Select Operator > expressions: f1 (type: string), demo_struct (type: > struct<f1:string,f2:string,f3:string>), '2019-01-01' (type: string) > outputColumnNames: _col0, _col1, _col2 > ListSink > set hive.cbo.enable=false; > explain select * from etltmp.test_struct where datestr='2019-01-01' and > demo_struct is not null; > STAGE PLANS: > Stage: Stage-0 > Fetch Operator > limit: -1 > Processor Tree: > TableScan > alias: test_struct > filterExpr: ((datestr = '2019-01-01') and demo_struct is not null) > (type: boolean) <----- Note that demo_struct filter is added when CBO is > turned off > Filter Operator > predicate: ((datestr = '2019-01-01') and demo_struct is not null) > (type: boolean) > Select Operator > expressions: f1 (type: string), demo_struct (type: > struct<f1:string,f2:string,f3:string>), '2019-01-01' (type: string) > outputColumnNames: _col0, _col1, _col2 > ListSink > {noformat} > In CalcitePlanner::genFilterRelNode, the following code misses to evaluate > this filter. > {noformat} > RexNode factoredFilterExpr = RexUtil > .pullFactors(cluster.getRexBuilder(), convertedFilterExpr); > {noformat} > Note that even if we add `demo_struct.f1` it would end up pushing the filter > correctly. -- This message was sent by Atlassian JIRA (v7.6.14#76016)