The GitHub Actions job "Flink CDC CI" on 
flink-cdc.git/FLINK-40640-iceberg-skip-redundant-update-delete has failed.
Run started by GitHub user dataengineervishal (triggered by lvyanquan).

Head commit for run:
67cc95a5a16e69a12d8661df6a2823e72c14ccca / Vishal 
<[email protected]>
[FLINK-41570][pipeline-connector/iceberg] Skip redundant before-image delete 
for UPDATEs that keep key and partition

Since FLINK-40640, every UPDATE on a table with equality fields is written
as DELETE(before) + INSERT(after). When neither an identifier field nor a
partition source column changes, the upsert INSERT already deletes the same
key in the same partition, so the explicit DELETE only adds a duplicate
equality-delete record.

IcebergWriter now caches, per table, getters for the target table's
identifier fields and partition source columns, and RowDataUtils skips the
before-image DELETE when none of them changed. Identifier fields are taken
from the target table rather than the CDC primary key, because they can
differ (transform primary-keys, pre-existing tables), in which case a key
change arrives as a plain UPDATE and still needs the DELETE.

Rows are mapped to Iceberg columns by position, so each column is resolved
by its position in the Iceberg schema rather than by name. This keeps the
comparison correct when the CDC and table column orders differ or columns
were renamed in the catalog. If a column is not top-level or its position
is outside the CDC schema, the DELETE is always kept.

Report URL: https://github.com/apache/flink-cdc/actions/runs/44588678068

With regards,
GitHub Actions via GitBox

Reply via email to