[
https://issues.apache.org/jira/browse/FLINK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser reopened FLINK-38574:
------------------------------------
Reopening for release-1.20. 9b21a9716cc went to master and was backported to
1.19, 2.0 and 2.1, but not to release-1.20. There, FLINK-40663 shows this
exception in all six of its failed jobs since 2026-09-24, each followed by the
checkpoint expiring ten minutes later:
https://github.com/apache/flink/actions/runs/36804869542/job/110195862980
{code}
03:20:39,784 [jobmanager-io-thread-4] WARN
org.apache.flink.runtime.jobmaster.JobMaster [] - Error while
processing AcknowledgeCheckpoint message
java.lang.IllegalStateException: Attempt to reference unknown state:
f71696db-7f3f-37f7-8d7e-0edbb2a2ba7c
at
org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
at
org.apache.flink.runtime.state.SharedStateRegistryImpl.registerReference(SharedStateRegistryImpl.java:97)
at
org.apache.flink.runtime.state.SharedStateRegistry.registerReference(SharedStateRegistry.java:53)
at
org.apache.flink.runtime.state.IncrementalRemoteKeyedStateHandle.registerSharedStates(IncrementalRemoteKeyedStateHandle.java:289)
{code}
The 1.19 commit bd916bd89e7 applies to release-1.20 without conflicts. We
should backport it there as well.
> Avoid reusing re-uploaded sst files when checkpoint notification is delayed
> ---------------------------------------------------------------------------
>
> Key: FLINK-38574
> URL: https://issues.apache.org/jira/browse/FLINK-38574
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.18.1, 1.19.3, 2.0.1, 1.20.3, 2.1.1
> Reporter: Zakelly Lan
> Assignee: Zakelly Lan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.4, 2.2.0, 2.1.2, 2.0.2
>
>
> It might be possible that from the TM's perspective, the checkpoint
> notification for last checkpoint is executed after the start of the next
> checkpoint, assuming no concurrent checkpoints. If so, this might indicate a
> bug. I'm saying that because this may make the
> `RocksIncrementalSnapshotStrategy` behaves wrong, diverging from the state
> tracking maintained by the `SharedStateRegistry` on the JM side. For example,
> considering a buggy event timeline for `RocksIncrementalSnapshotStrategy`:
> * Checkpoint 1 finished with state handle A for 1.sst.
> * Checkpoint 2 start, based on no checkpoint, so re-upload the 1.sst with
> handle B
> * Received Checkpoint 1's notification of finish.
> * Checkpoint 2 finished. JM thinks the handle A is subsumed.
> * Checkpoint 3 start, based on checkpoint 1, for 1.sst we reuse the handle A.
> * Received Checkpoint 1's subsume, Checkpoint 2's notification of finish.
> * Checkpoint 3 finished. JM received the handle A's placeholder and tell
> 'Attempt to reference unknown state', since the handle A is removed when
> checkpoint 2 finished.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
