tchivs opened a new pull request, #4523: URL: https://github.com/apache/flink-cdc/pull/4523
## What is the purpose of this pull request? Fix [FLINK-40560](https://issues.apache.org/jira/browse/FLINK-40560): a PostgreSQL snapshot-only source can finish successfully while leaving its replication slot behind, retaining WAL after the job is done. The stream split starts at the minimum snapshot high watermark and stops at the maximum. Its starting offset is subsequently advanced by emitted data-change records and heartbeats, not by every record consumed from WAL. Therefore, comparing the starting and ending offsets is not a reliable test of whether a bounded split has finished. ## Brief change log - Release the slot when a snapshot-only stream split finishes normally, using the reader context to distinguish completion from suspension for newly added tables. - Retain the slot when the stream split is suspended; do not move cleanup into `close()`, where it could destroy state needed for recovery. - Log a warning when `PostgresDialect.removeSlot` returns false. - Add reader-level tests for lagging starting offsets and suspended splits, plus a MiniCluster/PostgreSQL integration regression with multiple snapshot splits, no captured DML, and heartbeats disabled. --- ## Verifying this change This change added tests and can be verified as follows: - `PostgresSourceReaderTest`: all 8 tests pass, including both new lifecycle cases. - `PostgresSourceITCase#testSnapshotOnly*`: all 3 tests pass, including the new bounded-job regression. The regression reads all 21 fixture rows through end-of-input and then asserts that the replication slot is absent. - Mutation check: changing only the cleanup guard back to the original offset comparison makes the new integration test fail with `expected: 0L, but was: 1L` for the remaining slot count. Restoring the fix makes it pass. - Local verification used JDK 17 and the repository's default Flink 1.20.3 profile. The focused command below completed successfully; the configured unit and integration-test executions each ran the same 11 selected tests with no failures or skips. ```sh mvn -pl flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc \ com.diffplug.spotless:spotless-maven-plugin:apply mvn -pl flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc \ '-Dtest=PostgresSourceReaderTest,PostgresSourceITCase#testSnapshotOnly*' clean verify ``` Issue ownership: I have posted a claim comment on FLINK-40560. Self-assignment returned HTTP 403, so assignment to `tchivs` still requires a committer. ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes Generated-by: OpenAI ChatGPT via Oh My Pi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
