Feifan Wang created FLINK-30792:
-----------------------------------
Summary: clean up not uploaded state changes after materialization
complete
Key: FLINK-30792
URL: https://issues.apache.org/jira/browse/FLINK-30792
Project: Flink
Issue Type: Bug
Components: Runtime / State Backends
Affects Versions: 1.16.0
Reporter: Feifan Wang
We should clean up not uploaded state changes after materialization completed,
otherwise it will cause (status quo) :
# subsequent checkpoints contain wrong state changes which before completed
materialization
# FileNotFound exception may occur when recovering from the above problematic
checkpoint, because the state change files before completed materialization may
have been deleted with the checkpoint subsuming.
Since state changes before completed materialization in
FsStateChangelogWriter#notUploaded will not be used in any subsequent
checkpoint, I suggest clean up it while handle materialization result.
How do you think about this ? [~ym] , [~roman]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
