[
https://issues.apache.org/jira/browse/HIVE-25766?focusedWorklogId=689899&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-689899
]
ASF GitHub Bot logged work on HIVE-25766:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Dec/21 10:47
Start Date: 03/Dec/21 10:47
Worklog Time Spent: 10m
Work Description: asolimando commented on a change in pull request #2839:
URL: https://github.com/apache/hive/pull/2839#discussion_r761830232
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTransposeRule.java
##########
@@ -339,6 +339,11 @@ private void check(Filter filter) {
final RexNode filterCondition = simplify.simplify(filter.getCondition());
final Set<Integer> inputRefs =
HiveCalciteUtil.getInputRefs(newCondition);
+ // if the new IS NOT NULL has no input ref, there is redundancy here,
bail out
+ if (inputRefs.isEmpty()) {
Review comment:
For the `RelBuilder` use I think
[HiveFilterMergeRule](https://github.com/apache/hive/blob/cadf5f731c8cb0a4217d50de8a9b216eaaed1ea1/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterMergeRule.java#L54)
is a good example, and it hides all the complexity.
However, when dealing directly with newly created `RexNode`s I am not sure
how to achieve that consistently.
Regarding option 3, it seems that at least in the case reported here it was
not enough: https://github.com/apache/hive/pull/2840. My impression is that, if
we don't simplify before hand, in some cases, `RexSimplify.simplify` is not
always able to perform the simplification afterwards, but I need to have a
better look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 689899)
Time Spent: 1.5h (was: 1h 20m)
> java.util.NoSuchElementException in HiveFilterProjectTransposeRule if
> predicate has no InputRef
> -----------------------------------------------------------------------------------------------
>
> Key: HIVE-25766
> URL: https://issues.apache.org/jira/browse/HIVE-25766
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
> Affects Versions: 4.0.0
> Reporter: Alessandro Solimando
> Assignee: Alessandro Solimando
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The issue can be reproduced with the following query:
> {code:java}
> create table test1 (s string);
> create table test2 (m string);
> EXPLAIN CBO SELECT c.m
> FROM (
> SELECT substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) as m
> FROM test1
> WHERE substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) = '2') c
> JOIN test2 d ON c.m = d.m;
> {code}
> It fails with the following exception:
> {noformat}
> java.util.NoSuchElementException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447)
> at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303)
> at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
> at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316)
> at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191){noformat}
> The current implementation, while checking if the predicate to be transposed
> is redundant or not, it expects at least one InputRef, but the predicate can
> have none as in this case.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
