lucasgameiroborges opened a new pull request, #18101: URL: https://github.com/apache/iceberg/pull/18101
## What `RowDataConverter.convert()` rebuilds a record whenever its schema is not byte-identical to the target table's — a column-order difference, a missing optional column, a widening type. It builds the replacement as: ```java GenericRowData targetData = new GenericRowData(fieldGetters.length); ``` `GenericRowData(int)` defaults `RowKind` to `INSERT`, and nothing downstream restores the original kind — `RowKind` is never referenced anywhere else in the `sink/dynamic` package. ## Why `BaseDeltaTaskWriter` dispatches purely on `RowKind`. So a CDC delete/`UPDATE_BEFORE` that happens to go through this conversion path (e.g. a delete envelope carrying only key columns) is treated as an `INSERT`: - against a table with a **required** non-key column → `NullPointerException` downstream - without one → silently written as an insert of a mostly-null row instead of a delete Everything else the converter does is already correct — it maps fields by name, null-fills missing optional columns, and widens types — so carrying the kind through is the whole fix. ## Fix ```java GenericRowData targetData = new GenericRowData(sourceData.getRowKind(), fieldGetters.length); ``` Added `TestRowDataConverter#testPreservesRowKind`, which fails on the old code (`expected: DELETE but was: INSERT`) and passes with the fix. Applied to `flink/v2.3` only, following the convention used for the original RowData-evolution feature (#13340, backported to v1.19/v1.20 separately in #13401) — happy to backport to v1.20/v2.1/v2.2 in a follow-up if maintainers want it there too. Closes #18100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
