thexiay commented on PR #8367: URL: https://github.com/apache/inlong/pull/8367#issuecomment-1612703587
问题场景： 1.假设当前chk为n，n对应的iceberg snapshot已经提交了，max-committed-chk-id为n 2.手动回滚任务到chk为m（m<n）的位点 3.在checkpointId追赶到max-committed-chk-id之前，新snapshot都不会提交，所有uncommited manifest缓存在state之中 4.此时一旦出现runtime exception，任务重启，restore时尝试从state恢复uncommited manifest，但是由于会和max-committed-chk-id比较，只有大于max-committed-chk-id才会提交，所有uncommited manifest被裁剪了，数据丢失。 详细可参考测试用例`TestRollbackAndRecover#testRecoveryFromSnapshotWithoutCompletedNotification` 解决思路： 1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
