[
https://issues.apache.org/jira/browse/FLINK-30792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683642#comment-17683642
]
Yanfei Lei commented on FLINK-30792:
------------------------------------
[~Feifan Wang] Thanks for the investigation, after discussing offline, this
case would indeed result in FileNotFound.
As described in
https://issues.apache.org/jira/browse/FLINK-28440?focusedCommentId=17682912&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17682912
, this will cause FileNotFound may be encountered in all checkpoints after
chk-4 and before the next materialization.
We are going to use activeSqn directly during pre-emptive to avoid changesets
that have been materialized but not uploaded from being uploaded and then
deleted.
!image-2023-02-03-11-30-40-198.png|width=661,height=155!
> clean up not uploaded state changes after materialization complete
> ------------------------------------------------------------------
>
> Key: FLINK-30792
> URL: https://issues.apache.org/jira/browse/FLINK-30792
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.16.0
> Reporter: Feifan Wang
> Priority: Major
> Attachments: image-2023-02-03-11-30-40-198.png
>
>
> We should clean up not uploaded state changes after materialization
> completed, otherwise it may cause FileNotFoundException.
> Since state changes before completed materialization in
> FsStateChangelogWriter#notUploaded will not be used in any subsequent
> checkpoint, I suggest clean up it while handle materialization result.
> How do you think about this ? [~ym] , [~roman]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
