SEZ9 commented on issue #12565: URL: https://github.com/apache/seatunnel/issues/12565#issuecomment-5946894190
Thanks for the detailed report. Following up on the earlier analysis: after a savepoint restore with newly added tables, the merged incremental split's start offset and per-table watermark filter appear to be derived only from the newly snapshotted tables, so tables that were already in the pure-increment phase can lose changes between the saved offset and the new tables' snapshot high watermark. Looking at current `dev` (`53733a112c9e935c5d97f7f24be249caa43b4fd5`), `IncrementalSourceStreamFetcher.configureFilter()` rebuilds the per-table high-watermark state from `completedSnapshotSplitInfos`, and when that set is empty a single startup offset is applied to every captured table. That means the actual serialized incremental-split contents are what will decide whether this is a split-state reconstruction bug or something connector/sink specific — the job config alone can't tell us which old-table records were filtered once the table list changed. To move this forward, could you provide a minimal MySQL CDC regression case covering: 1. The exact commit you reproduced on. 2. The saved `IncrementalSplit` after restore: table IDs, startup offset, and the completed-snapshot split high watermarks. 3. The table-list change you applied between savepoint and restore. 4. Ordered source and sink records for one old table and the newly added table across the restore window, with an assertion showing an old-table change after the saved offset is dropped while the new-table snapshot succeeds. With that in hand we can confirm the root cause in the split-state reconstruction and define a testable acceptance boundary for the fix. <!-- streview-comment:1473 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
