[
https://issues.apache.org/jira/browse/FLINK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lincoln lee updated FLINK-29225:
--------------------------------
Description:
Currently if the interval between the arrival of the delete message and the
insert/update message exceeds state ttl, the delete message was ignored
incorrectly in `SinkUpsertMaterializer`. This will cause wrong result in
corresponding sink table(dirty data left).
1. if state ttl is set to '10 hour', then the following delete message will be
ignored (the '+I (1, a1)' will be left in sink table forever)
{code:java}
00:00:01 +I (1, a1)
10:00:02 -D (1, a1)
{code}
2. but another contrast case which will wrongly delete data in sink table if we
send delete message when state staled
{code}
00:00:01 +I (1, a1)
00:00:02 +I (1, a2)
10:00:03 -D (1, a1)
{code}
was:
Currently if the interval between the arrival of the delete message and the
insert/update message exceeds state ttl, the delete message was ignored
incorrectly in `SinkUpsertMaterializer`. This will cause wrong result in
corresponding sink table(dirty data left).
{code}
+I (1, a1)
-D (1, a1)
{code}
> Delete message incorrectly ignored in SinkUpsertMaterializer
> ------------------------------------------------------------
>
> Key: FLINK-29225
> URL: https://issues.apache.org/jira/browse/FLINK-29225
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.14.5, 1.15.2
> Reporter: lincoln lee
> Priority: Major
>
> Currently if the interval between the arrival of the delete message and the
> insert/update message exceeds state ttl, the delete message was ignored
> incorrectly in `SinkUpsertMaterializer`. This will cause wrong result in
> corresponding sink table(dirty data left).
>
> 1. if state ttl is set to '10 hour', then the following delete message will
> be ignored (the '+I (1, a1)' will be left in sink table forever)
> {code:java}
> 00:00:01 +I (1, a1)
> 10:00:02 -D (1, a1)
> {code}
>
> 2. but another contrast case which will wrongly delete data in sink table if
> we send delete message when state staled
> {code}
> 00:00:01 +I (1, a1)
> 00:00:02 +I (1, a2)
> 10:00:03 -D (1, a1)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
