zhangyue19921010 opened a new issue, #11344: URL: https://github.com/apache/paimon/issues/11344
### Search before asking - [x] I searched the issues and pull requests and found no matching reader-progress reset report. ### Paimon version master, `ad4f6e09ef51c6b9932cbf22f0be563c6dde683f` (2.2-SNAPSHOT). ### Compute Engine Flink 1.20.1 source connector. Reproduced with a real Paimon append table and consumer file through the source enumerator APIs on JDK 11; no full Flink job was run. ### Minimal reproduce step 1. Create an ordinary append table with `consumer-id` and `scan.mode=from-snapshot`, `scan.snapshot-id=1`, and commit three snapshots. 2. Complete a checkpoint while a reader is assigned snapshot 1. Its consumer progress is 1. 3. Let the reader finish snapshots 1 and 2 and receive snapshot 3. An additional checkpoint may be captured but not completed. 4. Reset the reader to the completed checkpoint. Flink returns only the later assignments to the enumerator, while the reader restores its snapshot-1 split separately. The restored reader does not request another split while replaying that split. 5. Complete a new checkpoint during replay. The current source writes consumer progress 3, although the restored reader still needs snapshot 1. The [regressions](https://github.com/zhangyue19921010/flink-table-store/blob/47063820395755e7ce9cc7c02da186c752208daa/paimon-flink/paimon-flink-common/src/test/java/org/apache/paimon/flink/source/ConsumerProgressFailoverTest.java) cover returned and empty assignments, readers that had requested another split, and older pending checkpoints. The two original reader-reset cases fail on unchanged production code with consumer progress 3 instead of 1. ### What doesn't meet your expectations? Consumer progress must not move past splits still needed by a restored reader. The old reader's assigned/consumed progress, pending split request and checkpoint minima survive `addSplitsBack`, so a later checkpoint can publish progress calculated before the reset. ### Anything else? Reset the affected reader's progress and pending split request, and discard pending checkpoint minima when assignments are returned. Fresh reader progress then advances the consumer normally. This is separate from the dedicated-compaction expired-checkpoint fallback in #9535; it does not change scan startup or expired-snapshot recovery policy. ### Are you willing to submit a PR? - [x] I am willing to submit a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
