Eric Pangiawan created KAFKA-15838: -------------------------------------- Summary: [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields Key: KAFKA-15838 URL: https://issues.apache.org/jira/browse/KAFKA-15838 Project: Kafka Issue Type: Bug Reporter: Eric Pangiawan
ExtractField: Line 116-119 https://github.com/a0x8o/kafka/blob/master/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/ExtractField.java#L61-L68 InsertField: Line 163 - 195 [https://github.com/a0x8o/kafka/blob/master/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/InsertField.java#L163-L195] h1. Expect: Value `null` is valid for an optional filed, even though the field has a default value. Only when field is required, the class return default value fallback when value is `null`. h1. Actual: Always return default value if `null` was given. h1. Example: PostgreSQL DDL: {code:java} CREATE TABLE products( id varchar(255), color varchar(255), quantity float8 ); -- Set Default ALTER TABLE products ALTER COLUMN quantity SET DEFAULT 1.0; {code} Insert A Record: {code:java} INSERT INTO public.products VALUES('1', 'Blue', null); {code} Table Select *: {code:java} id | color | quantity ----+-------+---------- 1 | Blue | {code} Debezium Behavior when using ExtractField and InsertField class (in the event flattening SMT): {code:java} { "id":"1", "color":"Blue", "quantity":1.0, "__op":"c", "__ts_ms":1698127432079, "__source_ts_ms":1698127431679, "__db":"testing_db", "__schema":"public", "__table":"products", "__lsn":24470112, "__txId":957, "__snapshot":null, "__deleted":"false" } {code} The debezium code can be found [here|https://github.com/debezium/debezium/blob/2.4/debezium-core/src/main/java/io/debezium/transforms/ExtractNewRecordState.java#L116-L119] h1. Expected Output: {code:java} { "id":"1", "color":"Blue", "quantity":null, "__op":"c", "__ts_ms":1698127432079, "__source_ts_ms":1698127431679, "__db":"testing_db", "__schema":"public", "__table":"products", "__lsn":24470112, "__txId":957, "__snapshot":null, "__deleted":"false" } {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)