fatmanverse commented on PR #11487: URL: https://github.com/apache/seatunnel/pull/11487#issuecomment-6552184810
While re-checking this PR against the latest `dev`, I found one more restore edge case and fixed it in 7cffa0f13. The snapshot-only restore keeps the checkpointed table set, as requested earlier. But if a table still had unread snapshot data and was then removed from `table-names` (or dropped) before restoring from the savepoint, the enumerator still split it while the readers no longer captured it. Debezium's `tableFor` returns null for a table outside `table.include.list`, so `MySqlSnapshotSplitReadTask` failed with a `NullPointerException`, again on every restart from the same checkpoint. `IncrementalSource#restoreEnumerator` now rejects such a restore up front, with an error naming the tables. It checks only tables that still have unread data: tables that are not split yet, remaining splits, and assigned splits without a completion watermark. A table whose splits have all finished is not read again, so removing or dropping it does not block the restore. Matching against the configured tables ignores case and any catalog/schema the checkpointed id lacks, so a table is only rejected when it is certainly not configured. The checkpointed table set itself is unchanged, and so is discovery. New cases in `IncrementalSourceTest`: an unread table that is no longer configured, one configured only in another schema, a finished table that is no longer configured, and MySQL/PostgreSQL table id shapes. The two rejection cases fail without the fix. I also added the restore behavior to the snapshot-only sections of the MySQL-CDC and PostgreSQL-CDC docs, and a short note to the OpenGauss-CDC docs (en/zh). The PR description is updated for the current scope as well. Verified on JDK 17 with this branch merged into the latest `dev` (f3e1c2ee3): `connector-cdc-base` 141, `connector-cdc-mysql` 59, `connector-cdc-oceanbase` 10, `connector-cdc-postgres` 93 and `connector-cdc-opengauss` 7 unit tests, 0 failures; `MysqlCDCSnapshotOnlyIT` 5/5; `spotless:check` passes. I know this PR is scheduled for after 3.1.0; this just keeps it ready for then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
