[
https://issues.apache.org/jira/browse/FLINK-28566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-28566:
-----------------------------------
Labels: pull-request-available (was: )
> Adds materialization support to eliminate the non determinism generated by
> lookup join node
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-28566
> URL: https://issues.apache.org/jira/browse/FLINK-28566
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
> Reporter: lincoln lee
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.16.0
>
>
> In order to minimize the potential exceptions or data errors when many users
> use the update stream to lookup join an external
> table (essentially due to the non-deterministic result based on
> processing-time to lookup external tables).
> When update exists in the input stream and the lookup key does not contain
> the primary key of the external table,
> FLINK automatically adds materialization of the update by default, so that it
> will only lookup the external table
> when the insert or update_after message arrives, and when the delete or
> update_before message arrives, it will
> directly querying the latest version of the locally materialized data and
> sent it to downstream operator.
> To do so，we introduce a new option
> 'table.exec.lookup-join.upsert-materialize' and resue the
> `UpsertMaterialize`. By default, the materialize operator will be added when
> an update stream lookup an external table without containing its primary
> keys(includes no primary key defined). You can also choose no
> materialization(NONE) or force materialization(FORCE) which will always
> enable materialization except input is insert only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
