huyuanfeng2018 opened a new issue, #4998: URL: https://github.com/apache/paimon/issues/4998
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 0.8+ ### Compute Engine Flink ### Minimal reproduce step 1. Use cdc action and do not configure checkpoint intervals (Paimon will actively configure checkpoint via: #2461) 2. open `commit.force-create-snapshot` = `true` 3. Actively kill a taskmanager during a task run, triggering a flink failover. ### What doesn't meet your expectations? ## Exception Chances are, we'll find that the task keeps restarting. ## How this led to this Exception - With `commit.force-create-snapshot` turned on, a commit will be triggered to generate a snapshot of paimon even if no data is written. - flink's failover resumes from checkpoint, based on paimon's two-phase commit, and resuming from checkpoint may trigger a commit of the manifest file - If there are no new data files in the commit (via the streamingCheckpointEnabled variable in the `CommitterOperator`), the manifest file commit may be filtered out because we didn't actively set the checkpoint. `streamingCheckpointEnabled= false`, then the commit will be ignored (because no new data file has been generated). Then, after the second job restart, the process will be repeated again, and we will be stuck in a dead loop. ### Anything else? _No response_ ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
