[
https://issues.apache.org/jira/browse/FLINK-30792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681924#comment-17681924
]
Hangxiang Yu commented on FLINK-30792:
--------------------------------------
IIUC, notuploaded will be truncated after materialization competed and
checkpoint subsuming currently in theory ?
Do you have seen some exceptions that could be shared ?
> clean up not uploaded state changes after materialization complete
> ------------------------------------------------------------------
>
> Key: FLINK-30792
> URL: https://issues.apache.org/jira/browse/FLINK-30792
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.16.0
> Reporter: Feifan Wang
> Priority: Major
>
> We should clean up not uploaded state changes after materialization
> completed, otherwise it will cause (status quo) :
> # subsequent checkpoints contain wrong state changes which before completed
> materialization
> # FileNotFound exception may occur when recovering from the above
> problematic checkpoint, because the state change files before completed
> materialization may have been deleted with the checkpoint subsuming.
> Since state changes before completed materialization in
> FsStateChangelogWriter#notUploaded will not be used in any subsequent
> checkpoint, I suggest clean up it while handle materialization result.
> How do you think about this ? [~ym] , [~roman]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
