[jira] [Updated] (FLINK-35313) Add upsert changelog mode to avoid UPDATE_BEFORE records push down
[
https://issues.apache.org/jira/browse/FLINK-35313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ude updated FLINK-35313:
Attachment: (was: image-2024-05-09-15-02-33-572.png)
> Add upsert changelog mode to avoid UPDATE_BEFORE records push down
> --
>
> Key: FLINK-35313
> URL: https://issues.apache.org/jira/browse/FLINK-35313
> Project: Flink
> Issue Type: New Feature
> Components: Flink CDC
>Reporter: ude
>Priority: Major
> Labels: pull-request-available
>
> I try to use flink sql to write mysql cdc-data into redis as a dimension
> table for other business use. When executing {{UPDATE}} DML, the cdc-data
> will be converted into {{-D (UPDATE_BEFORE)}} and {{+I (UPDATE_AFTER)}} two
> records to sink redis. However, delete first will cause other data streams to
> be lost(NULL) when join data, which is unacceptable.
> I think we can add support for [upser changelog
> mode|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/concepts/dynamic_tables/#table-to-stream-conversion]
> by adding changelogMode option with mandatory primary key
> configuration.Basically, with {{changelogMode=upsert}} we will avoid
> {{UPDATE_BEFORE}} rows and we will require a primary key for the table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35313) Add upsert changelog mode to avoid UPDATE_BEFORE records push down
[
https://issues.apache.org/jira/browse/FLINK-35313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-35313:
---
Labels: pull-request-available (was: )
> Add upsert changelog mode to avoid UPDATE_BEFORE records push down
> --
>
> Key: FLINK-35313
> URL: https://issues.apache.org/jira/browse/FLINK-35313
> Project: Flink
> Issue Type: New Feature
> Components: Flink CDC
>Reporter: ude
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2024-05-09-15-02-33-572.png
>
>
> I try to use flink sql to write mysql cdc-data into redis as a dimension
> table for other business use. When executing {{UPDATE}} DML, the cdc-data
> will be converted into {{-D (UPDATE_BEFORE)}} and {{+I (UPDATE_AFTER)}} two
> records to sink redis. However, delete first will cause other data streams to
> be lost(NULL) when join data, which is unacceptable.
> I think we can add support for [upser changelog
> mode|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/concepts/dynamic_tables/#table-to-stream-conversion]
> by adding changelogMode option with mandatory primary key
> configuration.Basically, with {{changelogMode=upsert}} we will avoid
> {{UPDATE_BEFORE}} rows and we will require a primary key for the table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35313) Add upsert changelog mode to avoid UPDATE_BEFORE records push down
[
https://issues.apache.org/jira/browse/FLINK-35313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ude updated FLINK-35313:
Attachment: image-2024-05-09-15-02-33-572.png
> Add upsert changelog mode to avoid UPDATE_BEFORE records push down
> --
>
> Key: FLINK-35313
> URL: https://issues.apache.org/jira/browse/FLINK-35313
> Project: Flink
> Issue Type: New Feature
> Components: Flink CDC
>Reporter: ude
>Priority: Major
> Attachments: image-2024-05-09-15-02-33-572.png
>
>
> I try to use flink sql to write mysql cdc-data into redis as a dimension
> table for other business use. When executing {{UPDATE}} DML, the cdc-data
> will be converted into {{-D (UPDATE_BEFORE)}} and {{+I (UPDATE_AFTER)}} two
> records to sink redis. However, delete first will cause other data streams to
> be lost(NULL) when join data, which is unacceptable.
> I think we can add support for [upser changelog
> mode|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/concepts/dynamic_tables/#table-to-stream-conversion]
> by adding changelogMode option with mandatory primary key
> configuration.Basically, with {{changelogMode=upsert}} we will avoid
> {{UPDATE_BEFORE}} rows and we will require a primary key for the table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
