SEZ9 commented on issue #12565: URL: https://github.com/apache/seatunnel/issues/12565#issuecomment-6460543330
Thanks @goutamadwant for opening #13006 as the focused replacement for #12567 — that is exactly the path suggested above, and your description maps onto the restart-position points carried over from the earlier review: the per-table offset is kept by the enumerator, passed through the rebuilt incremental split, honored by the exactly-once filter, and retained in the reader checkpoint until the stream has read past it, with unit coverage for both missing rows and no replay before a table's own offset when two restored splits at different offsets are regrouped. The before/after MySQL numbers (109/129 on `dev` vs 129/129 with the fix, and no duplicates in the append-only copy) are a useful end-to-end signal, and the note that the e2e is limited to a single incremental split because restoring a MySQL savepoint with more than one is rejected is a reasonable explanation for why that part is covered at unit level. I will do the detailed review on #13006 itself rather than here. Two remaining asks so I can line it up: 1. Please make sure GitHub CI has run on the final head of #13006 and request a review there once it has a result — I have not verified the current CI state of that PR in this thread, so I am not asserting one. 2. Please point out in the PR which unit tests cover the `incremental.parallelism > 1` missing/duplicate cases, so the regrouped-splits behavior can be checked against the exact tests. @WXPNG, as goutamadwant suggested, rechecking your original old-table/new-table row counts against #13006 would give us an independent acceptance signal on top of the numbers above. Until #13006 is merged, this issue stays open; `dev` at `4f5e3293740` still does not contain the per-table restore-offset fix. <!-- streview-comment:2094 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
