[
https://issues.apache.org/jira/browse/FLINK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan updated FLINK-38574:
--------------------------------------
Fix Version/s: 1.19.4
2.1.2
2.0.2
> Avoid reusing re-uploaded sst files when checkpoint notification is delayed
> ---------------------------------------------------------------------------
>
> Key: FLINK-38574
> URL: https://issues.apache.org/jira/browse/FLINK-38574
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.18.1, 1.19.3, 2.0.1, 1.20.3, 2.1.1
> Reporter: Zakelly Lan
> Assignee: Zakelly Lan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.4, 2.2.0, 2.1.2, 2.0.2
>
>
> It might be possible that from the TM's perspective, the checkpoint
> notification for last checkpoint is executed after the start of the next
> checkpoint, assuming no concurrent checkpoints. If so, this might indicate a
> bug. I'm saying that because this may make the
> `RocksIncrementalSnapshotStrategy` behaves wrong, diverging from the state
> tracking maintained by the `SharedStateRegistry` on the JM side. For example,
> considering a buggy event timeline for `RocksIncrementalSnapshotStrategy`:
> * Checkpoint 1 finished with state handle A for 1.sst.
> * Checkpoint 2 start, based on no checkpoint, so re-upload the 1.sst with
> handle B
> * Received Checkpoint 1's notification of finish.
> * Checkpoint 2 finished. JM thinks the handle A is subsumed.
> * Checkpoint 3 start, based on checkpoint 1, for 1.sst we reuse the handle A.
> * Received Checkpoint 1's subsume, Checkpoint 2's notification of finish.
> * Checkpoint 3 finished. JM received the handle A's placeholder and tell
> 'Attempt to reference unknown state', since the handle A is removed when
> checkpoint 2 finished.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
