[
https://issues.apache.org/jira/browse/HIVE-29121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18012523#comment-18012523
]
Seonggon Namgung commented on HIVE-29121:
-----------------------------------------
Thanks for your kind and detailed explanation. Let me try replacing
HiveSemiJoin with HiveJoin and check whether that resolves our issue. (At this
point, I believe it's worth doing regardless of our issue, although it will
take lower priority for me if it doesn't help us.)
Regarding the divergence issue, I've quickly reviewed Calcite's version and
confirmed that the proposed patch doesn't bring the two implementations any
closer.
> Restore HiveSubQueryRemoveRule to use InnerJoin instead of SemiJoin for
> uncorrelated IN/EXISTS subqueries with RelOptUtil.Logic.TRUE.
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-29121
> URL: https://issues.apache.org/jira/browse/HIVE-29121
> Project: Hive
> Issue Type: Improvement
> Environment: [^plan.example.txt]
> Reporter: Seonggon Namgung
> Assignee: Seonggon Namgung
> Priority: Major
> Labels: pull-request-available
> Attachments: plan.example.txt
>
>
> This JIRA is an addendum patch to HIVE-24685 and aims to restore the compiler
> logic from HIVE-17767.
> During the substitution of HiveSubQRemoveRelBuilder with Calcite's RelBuilder
> in HIVE-24685, Hive was changed to always use SemiJoin when handling
> uncorrelated IN/EXISTS subqueries with logic == RelOptUtil.Logic.TRUE. Since
> the SemiJoin is intended for use with correlated IN/EXISTS subqueries in
> conjunction with AGGR removal (cf. HIVE-17767), we should avoid using
> SemiJoin for the uncorrelated case, which neither benefits from AGGR removal
> nor allows the application of rules that cannot handle HiveSemiJoin (e.g.,
> join reordering).
> For clarity, the following combinations of query plans are attached. From the
> attached plans, we can observe that HIVE-24685 introduces a SemiJoin without
> removing HiveAggregate, unlike HIVE-17767.
> The attached plans cover the following combinations:
> * {Before HIVE-17767, After HIVE-17767, After HIVE-24685}
> * {Correlated, Uncorrelated}
> * {Before subquery removal, After subquery removal, After decorrelation}
> We discovered this issue while investigating a performance regression in
> TPC-DS Query 23.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
