The GitHub Actions job "Flink CDC CI" on flink-cdc.git/FLINK-38647-postgres-changelog-mode-and-serializer has failed. Run started by GitHub user cansakiroglu (triggered by leonardBang).
Head commit for run: 48d85f2507ce95280f58522b653a24261f1bdeb4 / Mehmet Can Şakiroğlu <[email protected]> Support changelog-mode=upsert + null-tolerant DataChangeEventSerializer FLINK-38647 reports a NullPointerException from the Postgres Pipeline source when a captured table has REPLICA IDENTITY DEFAULT and an UPDATE arrives that does not change the primary key. Root cause: PostgresDataSource hardcodes DebeziumChangelogMode.ALL, which makes the deserializer extract a before-image that is null under DEFAULT replication, NPE'ing in DebeziumSchemaDataTypeInference.inferStruct. This change fixes the issue from two angles: 1. Connector side — expose 'changelog-mode' YAML option on the Postgres Pipeline connector (mirrors the legacy SQL connector). Accepts 'all' (default, current behaviour) or 'upsert'. In upsert mode the source emits UPDATE events with before == null and only after populated, so the pipeline runs cleanly under REPLICA IDENTITY DEFAULT without requiring FULL (which roughly multiplies WAL volume per UPDATE by column count). 2. Runtime side — make DataChangeEventSerializer null-tolerant. Three changes: - copy() null-guards each recordDataSerializer.copy() call so the chained CopyingChainingOutput.pushToOperator path stops NPE'ing on null before/after. - serialize() writes 2 leading boolean presence flags before conditionally writing each record. Required because the previous serialize() already skipped writing null fields, but deserialize() always tried to read two records back — the wire format itself couldn't roundtrip a null-before UPDATE. - deserialize() reads the 2 flag bytes and skips the corresponding read when absent. Wire format change is symmetric but breaking vs current 3.6.x. Production checkpoints/savepoints written by older versions cannot be restored by the new code without a snapshot-version-aware deserialize path; see the PR description for the proposed compat strategy (bumping DataChangeEventSerializerSnapshot.CURRENT_VERSION to 2 and reading old format when version == 1). Happy to add that in a follow-up commit on this PR — wanted reviewers' opinion on the strategy first. Signed-off-by: Mehmet Can Şakiroğlu <[email protected]> Report URL: https://github.com/apache/flink-cdc/actions/runs/26956727621 With regards, GitHub Actions via GitBox
