[
https://issues.apache.org/jira/browse/FLINK-30792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feifan Wang updated FLINK-30792:
--------------------------------
Description:
We should clean up not uploaded state changes after materialization completed,
otherwise it may cause FileNotFoundException.
Since state changes before completed materialization in
FsStateChangelogWriter#notUploaded will not be used in any subsequent
checkpoint, I suggest clean up it while handle materialization result.
How do you think about this ? [~ym] , [~roman]
was:
We should clean up not uploaded state changes after materialization completed,
otherwise it will cause (status quo) :
# subsequent checkpoints contain wrong state changes which before completed
materialization
# FileNotFound exception may occur when recovering from the above problematic
checkpoint, because the state change files before completed materialization may
have been deleted with the checkpoint subsuming.
Since state changes before completed materialization in
FsStateChangelogWriter#notUploaded will not be used in any subsequent
checkpoint, I suggest clean up it while handle materialization result.
How do you think about this ? [~ym] , [~roman]
> clean up not uploaded state changes after materialization complete
> ------------------------------------------------------------------
>
> Key: FLINK-30792
> URL: https://issues.apache.org/jira/browse/FLINK-30792
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.16.0
> Reporter: Feifan Wang
> Priority: Major
>
> We should clean up not uploaded state changes after materialization
> completed, otherwise it may cause FileNotFoundException.
> Since state changes before completed materialization in
> FsStateChangelogWriter#notUploaded will not be used in any subsequent
> checkpoint, I suggest clean up it while handle materialization result.
> How do you think about this ? [~ym] , [~roman]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
