[
https://issues.apache.org/jira/browse/FLINK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Paul updated FLINK-29225:
--------------------------------
Affects Version/s: 1.15.3
(was: 1.15.2)
> Delete message incorrectly ignored in SinkUpsertMaterializer
> ------------------------------------------------------------
>
> Key: FLINK-29225
> URL: https://issues.apache.org/jira/browse/FLINK-29225
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.14.5, 1.15.3
> Reporter: lincoln lee
> Priority: Major
>
> Currently if the interval between the arrival of the delete message and the
> insert/update message exceeds state ttl, the delete message was ignored
> incorrectly in `SinkUpsertMaterializer`. This will cause wrong result in
> corresponding sink table(dirty data left).
>
> 1. if state ttl is set to '10 hour', then the following delete message will
> be ignored (the '+I (1, a1)' will be left in sink table forever)
> {code:java}
> 00:00:01 +I (1, a1)
> 10:00:02 -D (1, a1)
> {code}
>
> 2. but another contrast case which will wrongly delete data in sink table if
> we send delete message when state staled
> {code:java}
> 00:00:01 +I (1, a1)
> 00:00:02 +I (1, a2)
> 10:00:03 -D (1, a1)
> {code}
>
> compare the two choice of current implementation and eager deletion, the
> former will cause dirty data left, but the later will cause some data
> lost(seems the former is less harmful..)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
