waterWang opened a new pull request, #17684: URL: https://github.com/apache/iceberg/pull/17684
Fixes #17652 ## Problem Kafka Connect's `Struct.get(Field)` returns the field's schema `defaultValue` when the stored value is `null`, without checking `isOptional()` (longstanding Connect behavior, see KAFKA-8713). The Iceberg sink and its bundled SMTs read field values with `get()` in several places, so an explicit `NULL` in the source ends up written to the Iceberg table as the column's default value. The most common trigger is Debezium CDC: for any column that is nullable with a non-NULL default (e.g. MySQL `VARCHAR(255) NULL DEFAULT ''`), every explicitly NULL value is replaced with the default. ## Fix Replace `Struct.get(Field)` with `Struct.getWithoutDefault(String)` at all affected call sites. `getWithoutDefault` returns the actual stored value without applying the schema default, so null values are preserved as null. ## Changes - `DebeziumTransform.java` — copy loop in `applyWithSchema` - `KafkaMetadataTransform.java` — copy loop in `applyWithSchema` - `CopyValue.java` — copy loop and single-field copy in `applyWithSchema` - `RecordConverter.java` — field reads in `structToIceberg`, `collectFieldNames`, and `structToVariant` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
