[
https://issues.apache.org/jira/browse/HIVE-28931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18081757#comment-18081757
]
Ruben Q L edited comment on HIVE-28931 at 5/18/26 2:50 PM:
-----------------------------------------------------------
[~zabetak] I have started the work in https://github.com/apache/hive/pull/6495.
For the moment, I have kept the "old"
{{RexNodeConverter#transformInToOrOperands}} call (instead of the new
SEARCH-generating {{RexNodeConverter#rewriteInClause}} method) in two places:
- {{JDBCExpandExpressionsRule#transformIntoOrAndClause}}: I assume in this
context Jdbc wouldn't know how to handle a SEARCH expression
- {{HiveInBetweenExpandRule#RexInBetweenExpander#visitCall}}: this shuttle
expands already SEARCH expressions, so I guess it does not make much sense to
convert IN => SEARCH in here
Apart from that, should we remove the (now unused)
{{RexNodeConverter#rewriteInClauseChildren}}? It's a public static method, so
it might still be used by downstream projects...
was (Author: rubenql):
[~zabetak] I have started the work in https://github.com/apache/hive/pull/6495.
For the moment, I have kept the "old"
{{RexNodeConverter#transformInToOrOperands}} call (instead of the new
SEARCH-generating {{RexNodeConverter#rewriteInClause}} method) in two places:
- {{JDBCExpandExpressionsRule#transformIntoOrAndClause}}: I assume in this
context Jdbc does not know how to handle a SEARCH expression
- {{HiveInBetweenExpandRule#RexInBetweenExpander#visitCall}}: this shuttle
expands already SEARCH expressions, so I guess it does not make much sense to
convert IN => SEARCH in here
Apart from that, should we remove the (now unused)
{{RexNodeConverter#rewriteInClauseChildren}}? It's a public static method, so
it might still be used by downstream projects...
> Convert IN to SEARCH in RexNodeConverter
> ----------------------------------------
>
> Key: HIVE-28931
> URL: https://issues.apache.org/jira/browse/HIVE-28931
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Reporter: Stamatis Zampetakis
> Assignee: Ruben Q L
> Priority: Major
> Labels: pull-request-available
>
> RexNodeConverter has some logic
> ([transformInToOrOperands|https://github.com/apache/hive/blob/4554364b94e60f10d3e4e723c8016ed3148b9198/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java#L520],
>
> [rewriteInClauseChildren|https://github.com/apache/hive/blob/4554364b94e60f10d3e4e723c8016ed3148b9198/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java#L570])
> for transforming IN to OR when certain conditions hold. The initial logic
> was introduced in HIVE-19097 to facilitate the simplification/unification of
> expressions (by RexSimplify) since the IN operator is mostly a blackbox for
> CBO so it cannot be easily combined with other equivalent expressions.
> In CALCITE-4173, the SEARCH operator was introduced to represent IN, BETWEEN,
> and various other expressions. The SEARCH operator is used extensively for
> simplifications and there is also some logic in RexSimplify that tries to
> transform OR to SEARCH when that's possible.
> After upgrading to Calcite version 1.33.0 (HIVE-27102), Hive also takes
> advantage of the SEARCH operator. Doing an early conversion from IN to SEARCH
> (instead of OR) in RexNodeConverter seems beneficial for enabling further
> simplifications and it could possibly reduce compilation latency since we
> would avoid the intermediate conversion to OR when that is not necessary. In
> a nutshell instead of doing IN -> OR -> SEARCH we could do IN -> SEARCH and
> we could possibly exploit the RexBuilder.makeIn method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
