guozhangwang opened a new pull request #8964: URL: https://github.com/apache/kafka/pull/8964
1. Do not always flush state stores when committing (in post-commit), move `stateMgr.flush` into post-commit to live with checkpointing. Then in post-commit, we checkpoint when: 1.a. The state store's snapshot has progressed much further compared to the previous checkpoint. 2.a. When the task is being closed, we enforce a snapshot (but as a minor optimization, we would avoid checkpointing if it is exactly the same). 2. However, for cache / suppression buffer, we still need to flush them in pre-commit to make sure all records sent via producers; I'd have to introduce a bit hacky workaround into the CachingStateStore in stateMgr. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
