[
https://issues.apache.org/jira/browse/FLINK-38854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Borisov updated FLINK-38854:
----------------------------------
Description:
In FLINK-38753, upsert keys are enriched for *binary joins* by propagating them
across equi-join key equivalences (via {{{}JoinInfo{}}}), using a bounded
fixpoint to cover cross-combinations of multiple equi predicates.
This logic currently does not apply when joins are represented as a
{{{}MultiJoin{}}}.
For {{{}MultiJoin{}}}, equivalence relations may span more than two inputs, and
upsert keys inferred from one input can be equivalent to columns originating
from other inputs. Since these substitutions are not derived today, the planner
may conservatively infer missing upsert keys and introduce unnecessary
{{{}upsertMaterialize{}}}.
*Expected Behavior*
For {{{}MultiJoin{}}}, the planner should:
* consider all equi-join key equivalence classes across inputs
* enrich inferred upsert keys by propagating them across these equivalences
* respect join null-generation semantics (outer joins)
* use a bounded fixpoint/worklist approach, similar to FLINK-38753
This is a planner metadata improvement only.
*Related*
* FLINK-38753 — Enrich upsert keys by equivalence expressions for binary joins
was:
*Description*
In FLINK-38753, upsert keys are enriched for *binary joins* by propagating them
across equi-join key equivalences (via {{{}JoinInfo{}}}), using a bounded
fixpoint to cover cross-combinations of multiple equi predicates.
This logic currently does not apply when joins are represented as a
{{{}MultiJoin{}}}.
For {{{}MultiJoin{}}}, equivalence relations may span more than two inputs, and
upsert keys inferred from one input can be equivalent to columns originating
from other inputs. Since these substitutions are not derived today, the planner
may conservatively infer missing upsert keys and introduce unnecessary
{{{}upsertMaterialize{}}}.
*Expected Behavior*
For {{{}MultiJoin{}}}, the planner should:
* consider all equi-join key equivalence classes across inputs
* enrich inferred upsert keys by propagating them across these equivalences
* respect join null-generation semantics (outer joins)
* use a bounded fixpoint/worklist approach, similar to FLINK-38753
This is a planner metadata improvement only.
h3. *Related*
* FLINK-38753 — Enrich upsert keys by equivalence expressions for binary joins
> Enrich upsert keys across equivalence relations in MultiJoin
> ------------------------------------------------------------
>
> Key: FLINK-38854
> URL: https://issues.apache.org/jira/browse/FLINK-38854
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Affects Versions: 2.2.0
> Reporter: Anton Borisov
> Priority: Major
>
> In FLINK-38753, upsert keys are enriched for *binary joins* by propagating
> them across equi-join key equivalences (via {{{}JoinInfo{}}}), using a
> bounded fixpoint to cover cross-combinations of multiple equi predicates.
> This logic currently does not apply when joins are represented as a
> {{{}MultiJoin{}}}.
> For {{{}MultiJoin{}}}, equivalence relations may span more than two inputs,
> and upsert keys inferred from one input can be equivalent to columns
> originating from other inputs. Since these substitutions are not derived
> today, the planner may conservatively infer missing upsert keys and introduce
> unnecessary {{{}upsertMaterialize{}}}.
> *Expected Behavior*
> For {{{}MultiJoin{}}}, the planner should:
> * consider all equi-join key equivalence classes across inputs
> * enrich inferred upsert keys by propagating them across these equivalences
> * respect join null-generation semantics (outer joins)
> * use a bounded fixpoint/worklist approach, similar to FLINK-38753
> This is a planner metadata improvement only.
> *Related*
> * FLINK-38753 — Enrich upsert keys by equivalence expressions for binary
> joins
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
