[
https://issues.apache.org/jira/browse/CALCITE-5365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128786#comment-18128786
]
Arnaud Jegou commented on CALCITE-5365:
---------------------------------------
The query in the description uses left joins, and `getAllPredicates` returns
null for an outer join, in 1.30.0 as on main, so as written it does not reach
`createAllPossibleExpressions`. With inner joins it does: `getAllPredicates`
builds 3^14 combinations for the seven-key join condition.
Most of those combinations cannot occur, which is now tracked in CALCITE-7869:
above a Union, lineage combines fields from different inputs. A row of the join
pairs one input of t1 with one input of t2, so only 3 x 3 = 9 combinations are
valid. On main, the same shape with two keys instead of seven returns 81
predicates instead of 9.
Fixing CALCITE-7869 would also address the concern above that a limit loses
predicate information: the valid combinations would stay small, so a limit
would only apply to plans that really have many source combinations, such as
joins of several wide unions. Splitting the condition into conjuncts, as
suggested above, would also avoid the product here, and stays useful after
CALCITE-7869: when the conjuncts relate different unions, as in a chain of
joins over several unions, the count becomes a sum instead of a product. The
two changes complement each other: CALCITE-7869 removes the combinations that
cannot occur, including within a single expression, and splitting keeps the
valid ones small.
> RelMdExpressionLineage#createAllPossibleExpressions will create many RexCall
> lead to OOM
> ------------------------------------------------------------------------------------------
>
> Key: CALCITE-5365
> URL: https://issues.apache.org/jira/browse/CALCITE-5365
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.30.0
> Reporter: hqbhoho
> Priority: Major
>
> Execute query like:
> {code:java}
> select
> *
> from
> (
> select *
> from
> (
> select * from A1
> )
> union all
> (
> select * from A2
> )
> union all
> (
> select * from A3
> )
> ) t1
> left join
> (
> select *
> from
> (
> select * from B1
> )
> union all
> (
> select * from B2
> )
> union all
> (
> select * from B3
> )
> )t2
> on t1.a1 = t2.b1
> and t1.a2 = t2.b2
> and t1.a3 = t2.b3
> and t1.a4 = t2.b4
> and t1.a5 = t2.b5
> and t1.a6 = t2.b6
> and t1.a7 = t2.b7
> left join C1 name
> on t2.b8=name.c1 {code}
> I will get all Predicates via RelMetadataQuery.getAllPredicates()，but will
> occur oom.
> RelMdExpressionLineage#createAllPossibleExpressions will retrun 3^14 RexCall
> when t1 join t2 with 7 condition? (t1.a1 will have 3 replacement)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
