li3zhi4 opened a new pull request, #11885: URL: https://github.com/apache/seatunnel/pull/11885
### Purpose of this pull request Fixes [issue #11884](https://github.com/apache/seatunnel/issues/11884): MySQL CDC `stop.mode = "latest"` silently drops every change written while the snapshot phase is running. **Root cause**: `IncrementalSplitAssigner` creates the incremental split once and calls `offsetFactory.latest()` at that moment, freezing the binlog position (job-start time) into `IncrementalSplit.stopOffset` (a `final` field). For snapshot-taking startups (`initial`/`earliest`) the snapshot phase advances the binlog well past that stale offset, so when the binlog phase starts `checkStopOffset()` immediately sees `currentBinlogOffset.isAtOrAfter(stopOffset) == true` and the job finishes without reading any post-snapshot change. **Fix**: resolve the effective stop offset **when the binlog phase starts** instead of at split-creation time. `MySqlBinlogFetchTask.execute()` now calls `MySqlConnectionUtils.currentBinlogOffset(connection)` for `stop.mode = latest` and passes it to `BoundedMySqlStreamingChangeEventSource`; `checkStopOffset()` prefers the dynamic offset for `latest`. `specific`/`timestamp` stop modes keep the split's configured offset — behavior unchanged. ### Does this PR introduce _any_ user-facing change? Yes: `stop.mode = "latest"` now terminates at the binlog position captured when the binlog phase begins (after the snapshot), so changes produced while the snapshot ran are no longer lost. For non-snapshot startups (`latest`/`specific`/`timestamp`) behavior is unchanged. ### How was this patch tested? - E2E: five startup modes (`initial`/`earliest`/`latest`/`specific`/`timestamp`) × `stop.mode = latest` (`testMysqlCdc{Initial,Earliest,Latest,Specific,Timestamp}StartupWithLatestStop`). - For `initial`/`earliest`, the test issues an `UPDATE` while the job is RUNNING and asserts the post-snapshot change (`f_varchar='latest-stop'`) is synced to the sink — this assertion **fails on the old stale stop offset** and passes with the fix. - Existing `specific`-stop tests still pass (no regression). - `spotless:check` ✅, unit tests ✅ (local). ### Check list - [x] Code changes complete and verified locally (5 startup modes × latest-stop e2e) - [x] Related unit tests pass - [x] New e2e tests added with data-boundary assertions - [x] `mvn spotless:check` passes - [ ] Docs updated if needed (behavior now matches the documented `stop.mode=latest` semantics) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
