jianguotian opened a new pull request, #9677: URL: https://github.com/apache/paimon/pull/9677
### Purpose `FlinkSourceBuilder` selects `MonitorSource` for streaming reads with a `consumer-id` in `EXACTLY_ONCE` mode. This path did not honor `scan.max-snapshot.count`, so it could continue planning non-empty snapshots before checkpoint completion confirmed progress for the snapshots already in flight. This PR makes the monitor source enforce the configured snapshot limit and ties snapshot-credit release to completed checkpoints. ### Changes - read `scan.max-snapshot.count` from the table options in the `MonitorSource` build path - track next-snapshot cursors for non-empty streaming plans while the limit is enabled - pause planning when the configured number of snapshots is in flight - release only credits covered by a completed checkpoint and wake the source reader - require checkpointing when a streaming monitor source enables the snapshot limit - reopen the transient planning gate after restore - keep the existing public constructor and Flink checkpoint state format unchanged The limit applies to concurrently planned snapshots. It does not cap the records or splits within one snapshot and does not change `scan.max-splits-per-task` semantics. ### Tests - [x] Flink 1.x module compile/install with Checkstyle, Spotless, Maven Enforcer, and RAT - [x] `OperatorSourceTest` with Flink 1.x: 10 tests passed - [x] `OperatorSourceTest` with Flink 2.x: 10 tests passed - [x] `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
