nzw921rx opened a new pull request, #13018:
URL: https://github.com/apache/seatunnel/pull/13018
### Purpose of this pull request
Related issue: #11889. Related PR: #11896.
Fix MongoDB CDC snapshot completion in non-exactly-once mode on the latest
`dev` and add regression coverage.
In non-exactly-once mode, the snapshot consumer stops at the HIGH watermark.
Continuing to enqueue backfill records can fill the bounded queue and prevent
the snapshot task from finishing. Skip backfill and the unused END watermark in
this mode, while preserving exactly-once backfill and merge behavior.
### Does this PR introduce _any_ user-facing change?
Yes. MongoDB CDC jobs with `exactly_once=false` can finish snapshots under
concurrent writes and continue reading incremental changes. Snapshot-period
changes are read from the low watermark in the incremental phase, so duplicate
records remain possible. English and Chinese documentation explain the
behavior. Configuration names, defaults, dependencies and public APIs remain
unchanged.
### How was this patch tested?
- Scoped Spotless formatting and E2E test compilation passed.
- All 51 MongoDB CDC unit tests passed with:
`./mvnw -pl seatunnel-connectors-v2/connector-cdc/connector-cdc-mongodb
package`
- The new unit regressions failed against unmodified `dev`, detecting the
unwanted END watermark and backfill task.
- Both new Docker/Zeta E2E cases passed, with zero failures, errors or
skipped tests. Each pauses the snapshot query with a MongoDB test failpoint,
applies 128 changes against a 32-slot queue, then verifies all 512 snapshot
records and subsequent insert, update and delete events.
```bash
./mvnw -pl
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mongodb-e2e -DskipTests
test-compile
RUN_ENGINES=zeta ./mvnw -pl
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mongodb-e2e
-DskipUT=true -DskipIT=false
'-Dit.test=MongodbCDCIT#*SnapshotWithConcurrentChanges'
failsafe:integration-test
```
The Failsafe XML report was checked directly. Maven `verify` was not run
because the workspace rules prohibit that goal.
### Check list
- [x] Unit and existing E2E test classes extended for the regression.
- [x] English and Chinese connector documentation updated.
- [x] New configuration file includes the ASF license header.
- No new dependencies, connector registrations or incompatible changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
