[
https://issues.apache.org/jira/browse/HIVE-28931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18082022#comment-18082022
]
Stamatis Zampetakis commented on HIVE-28931:
--------------------------------------------
I would consider RexNodeConverter an internal Hive API so I guess we can drop
the method since I doubt it is used by other projects and not sure if they
should use it anyways.
The HiveInBetweenExpandRule is used in the context of materialized view
rewritting. I assume that expanding IN/BETWEEN helps to apply more MV
transformations since the respective operators do not exist in Calcite
codebase. SEARCH on the other hand is used and widely present in Calcite so
maybe there is no need do this expansion. If for some reason SEARCH interferes
with MV rewritting then maybe its a bug in Calcite. Not sure if there is a
ticket about dropping the search expansion from HiveInBetweenExpandRule but if
not please log one and we can decide how to advance there.
The JDBCExpandExpressionsRule should not have SEARCH indeed.
> Convert IN to SEARCH in RexNodeConverter
> ----------------------------------------
>
> Key: HIVE-28931
> URL: https://issues.apache.org/jira/browse/HIVE-28931
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Reporter: Stamatis Zampetakis
> Assignee: Ruben Q L
> Priority: Major
> Labels: pull-request-available
>
> RexNodeConverter has some logic
> ([transformInToOrOperands|https://github.com/apache/hive/blob/4554364b94e60f10d3e4e723c8016ed3148b9198/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java#L520],
>
> [rewriteInClauseChildren|https://github.com/apache/hive/blob/4554364b94e60f10d3e4e723c8016ed3148b9198/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java#L570])
> for transforming IN to OR when certain conditions hold. The initial logic
> was introduced in HIVE-19097 to facilitate the simplification/unification of
> expressions (by RexSimplify) since the IN operator is mostly a blackbox for
> CBO so it cannot be easily combined with other equivalent expressions.
> In CALCITE-4173, the SEARCH operator was introduced to represent IN, BETWEEN,
> and various other expressions. The SEARCH operator is used extensively for
> simplifications and there is also some logic in RexSimplify that tries to
> transform OR to SEARCH when that's possible.
> After upgrading to Calcite version 1.33.0 (HIVE-27102), Hive also takes
> advantage of the SEARCH operator. Doing an early conversion from IN to SEARCH
> (instead of OR) in RexNodeConverter seems beneficial for enabling further
> simplifications and it could possibly reduce compilation latency since we
> would avoid the intermediate conversion to OR when that is not necessary. In
> a nutshell instead of doing IN -> OR -> SEARCH we could do IN -> SEARCH and
> we could possibly exploit the RexBuilder.makeIn method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
