li3zhi4 opened a new issue, #11884: URL: https://github.com/apache/seatunnel/issues/11884
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues) and found no existing issue reporting this bug: - [#11739](https://github.com/apache/seatunnel/issues/11739) — the audit issue for extending `stop.mode` to other CDC connectors (tracked by us; this bug is about MySQL's own `stop.mode=latest`). - [#11036](https://github.com/apache/seatunnel/issues/11036) — [Feature] MySQL-CDC snapshot-only startup mode; different concern (startup, not stop). - [#11847](https://github.com/apache/seatunnel/issues/11847) — PostgreSQL committed-offset restore skip; different connector. - No issue reports that MySQL CDC `stop.mode = "latest"` drops post-snapshot changes. ### What happened When MySQL CDC is configured with `stop.mode = "latest"` together with a snapshot-taking startup mode (`startup.mode = initial` or `earliest`), the job **terminates immediately after the snapshot phase** and silently **drops every change written while the snapshot was running**. Expected: `stop.mode = "latest"` means "read up to the current latest binlog position and then stop". After the snapshot phase completes, the binlog phase should keep reading from the snapshot-end position up to the binlog position at that moment — including all changes produced while the snapshot was in progress. Actual: the job reaches `FINISHED` right after the snapshot phase; the sink contains only the snapshot rows, and all DML performed during the snapshot window is lost. ### Why it happens `IncrementalSplitAssigner` creates the incremental split once and calls `StopConfig.getStopOffset(offsetFactory)` at that moment (`offsetFactory.latest()` → `MySqlConnectionUtils.currentBinlogOffset(...)`). The returned binlog position is **captured at job-start time** and frozen into `IncrementalSplit.stopOffset` (a `final` field). For `startup.mode = initial/earliest` the snapshot phase then runs and advances the binlog well past that frozen offset. When the binlog phase starts, `BoundedMySqlStreamingChangeEventSource.checkStopOffset()` immediately sees `currentBinlogOffset.isAtOrAfter(stopOffset) == true` and dispatches the END watermark — the job finishes without reading any post-snapshot change. Note: `stop.mode = "specific"` (implemented in [PR #11618](https://github.com/apache/seatunnel/pull/11618)) is unaffected — its stop offset is a user-configured fixed position. Only the dynamic `latest` mode is broken by the freeze-at-split-creation behavior. ### SeaTunnel Version Upstream `dev` at the time of investigation (2026-08-20); also reproducible on 2.3.13 (bug-020 locally). ### Steps to reproduce 1. Start a MySQL-CDC job with `startup.mode = "initial"` and `stop.mode = "latest"`. 2. While the job is RUNNING (snapshot still in progress), issue an `UPDATE`/`INSERT` on the source table. 3. Wait for the job to reach `FINISHED`. Observed: the change issued during the snapshot window is **not** present in the sink. Expected: it is present (it was written before the binlog phase began). ### Proposed fix (implemented locally) Resolve the effective stop offset **when the binlog phase starts** instead of at split-creation time: - In `MySqlBinlogFetchTask.execute()`, when `stop.mode = latest`, call `MySqlConnectionUtils.currentBinlogOffset(connection)` at binlog-phase start and pass it to `BoundedMySqlStreamingChangeEventSource` as the effective stop offset. - `checkStopOffset()` prefers the effective (dynamic) offset for `latest`; `specific`/`timestamp` keep the split's configured offset (behavior unchanged). - E2E: cover all five startup modes × `stop.mode=latest`; for `initial`/`earliest` assert the post-snapshot `UPDATE` is actually synced to the sink (this assertion fails without the fix). ### Related - PR: https://github.com/apache/seatunnel/pull/11618 (bounded-read infrastructure: `BoundedMySqlStreamingChangeEventSource`, `isBoundedReadFinished()`, `Offset.isNeverStop()`) - Issue: https://github.com/apache/seatunnel/issues/11739 (audit of stop-mode support across CDC connectors) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
