SEZ9 commented on PR #11988: URL: https://github.com/apache/seatunnel/pull/11988#issuecomment-5747517064
Summary of where this stands and what is still needed on the current head (`2fca2bef97`). **CI** - Fork run `34743368340` on `8e6039efe45e` was red only for reasons unrelated to this diff: the `minio/minio:RELEASE.2024-06-13T22-53-53Z` image is no longer available (Hudi S3 / Paimon S3 ITs) and a Canal-to-Pulsar data mismatch. None of those touch `connector-cdc-base` or MySQL CDC. - Correction to the earlier CI note: current `dev` already points those S3 ITs at the `quay.io/minio/minio` mirror; this branch just doesn't contain that change yet. - **Ask:** rebase onto current `dev` so we get an actually green run instead of one we have to explain away. **`IncrementalSplit#pruneTables()`** - Per-table start offsets now survive the prune and two regression tests were added. That was the one real bug here, and it was found by reading the restore path rather than by a test, which leads to the next point. **E2E coverage for `startup.mode` = `mixed` (blocking)** - MySQL CDC already has an IT suite, so per the contributor rules a new startup mode needs to be exercised there. Previously treated as recommended; now required. - **Ask:** add an IT that runs a job with `startup.mode` set to `mixed`, at least one table in `startup.snapshot-table-names` and at least one not listed, and asserts that (a) the listed table gets snapshot data plus subsequent binlog changes, (b) the unlisted table only receives changes at/after the configured `startup.specific-offset.*` position, and (c) this holds across a checkpoint stop/restore, since that is exactly where the pruning bug lived. **Smaller items** - Please confirm how the new mixed-mode policy in checkpoint state behaves when restoring from a checkpoint written before this change, or state explicitly that it isn't supported and document that. - A short docs entry for `mixed` and `startup.snapshot-table-names` in the MySQL CDC connector doc, including the interaction with `startup.specific-offset.*`, if not already included. Once the rebase and the `mixed` IT are in, I'll do a final pass on the new head. <!-- streview-comment:1193 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
