JNSimba opened a new pull request, #4367: URL: https://github.com/apache/flink-cdc/pull/4367
## Summary
This PR adds support for PostgreSQL tables without primary keys in
incremental snapshot mode,
following the same approach as MySQL (PR #2150).
### Changes
**JDBC Base Layer (flink-cdc-base):**
- `SplitKeyUtils.getSplitKey()`: Handle null record key by extracting
chunk key from value's
after/before struct instead of record key
- `IncrementalSourceScanFetcher.pollWithBuffer()`: Use after struct as
buffer key when
record key is null (no-PK table)
- `JdbcSourceFetchTaskContext.rewriteOutputBuffer()`: Support no-PK merge
logic — for tables
without primary key, use before/after struct as buffer key for
CREATE/UPDATE/DELETE operations
**PostgreSQL Connector (flink-connector-postgres-cdc):**
- `PostgresDialect`: Validate that tables without primary key must have
`REPLICA IDENTITY FULL`
set, querying `pg_class.relreplident` and failing fast with a clear
error message if not
**Tests:**
- Added `products_no_pk` table DDL (without PK, with `REPLICA IDENTITY
FULL`)
- `testNoPKTableWithChunkKey`: End-to-end test for no-PK table with
`scan.incremental.snapshot.chunk.key-column` configured
- `testNoPKTableWithoutChunkKey`: Verify `ValidationException` is thrown
when chunk key column is not specified
### Design Decisions
- **No interface change**: Kept `Map<Struct, SourceRecord>` in
`FetchTask.Context.rewriteOutputBuffer()`
unchanged to avoid breaking MongoDB and other connectors. For no-PK
tables, the full row struct
(all columns via REPLICA IDENTITY FULL) serves as the buffer key.
- **At-least-once for chunk key updates**: When an UPDATE changes the
chunk key column value and
crosses split boundaries, only at-least-once semantics can be guaranteed
(consistent with MySQL behavior).
- **REPLICA IDENTITY FULL required**: PostgreSQL requires this setting on
no-PK tables to provide
full before image in WAL for UPDATE/DELETE events. The connector
validates this at startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
