[
https://issues.apache.org/jira/browse/FLINK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18045572#comment-18045572
]
Roman Khachatryan commented on FLINK-38574:
-------------------------------------------
Merged
to 1.19 as bd916bd89e739504f63debb4d155a709f3b0569b
to 2.0 as a8dac03c829d877cbfd448d6b4add39283105452
to 2.1 as 9f6cff3a78f71b5422178e970dae758ce03868ee
> Avoid reusing re-uploaded sst files when checkpoint notification is delayed
> ---------------------------------------------------------------------------
>
> Key: FLINK-38574
> URL: https://issues.apache.org/jira/browse/FLINK-38574
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.18.1, 1.19.3, 2.0.1, 1.20.3, 2.1.1
> Reporter: Zakelly Lan
> Assignee: Zakelly Lan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.4, 2.2.0, 2.1.2, 2.0.2
>
>
> It might be possible that from the TM's perspective, the checkpoint
> notification for last checkpoint is executed after the start of the next
> checkpoint, assuming no concurrent checkpoints. If so, this might indicate a
> bug. I'm saying that because this may make the
> `RocksIncrementalSnapshotStrategy` behaves wrong, diverging from the state
> tracking maintained by the `SharedStateRegistry` on the JM side. For example,
> considering a buggy event timeline for `RocksIncrementalSnapshotStrategy`:
> * Checkpoint 1 finished with state handle A for 1.sst.
> * Checkpoint 2 start, based on no checkpoint, so re-upload the 1.sst with
> handle B
> * Received Checkpoint 1's notification of finish.
> * Checkpoint 2 finished. JM thinks the handle A is subsumed.
> * Checkpoint 3 start, based on checkpoint 1, for 1.sst we reuse the handle A.
> * Received Checkpoint 1's subsume, Checkpoint 2's notification of finish.
> * Checkpoint 3 finished. JM received the handle A's placeholder and tell
> 'Attempt to reference unknown state', since the handle A is removed when
> checkpoint 2 finished.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
