mengyejiang opened a new issue, #11068: URL: https://github.com/apache/paimon/issues/11068
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version ### Paimon version 1.2.0 ### Engine and environment - Apache Flink 1.16.2 - Java 8 - Kafka source containing Oracle GoldenGate (OGG) JSON messages - Paimon action: kafka_sync_database ### Description When ingesting OGG JSON messages, JSON null values are converted to the literal string "null" during record parsing. For a STRING column, the value is therefore stored as the string "null" instead of SQL NULL. The affected code is AbstractJsonRecordParser.extractRowData(), which is used by OggRecordParser. ### Minimal example An OGG insert message: { "table": "TEST.CUSTOMER", "op_type": "I", "op_ts": "2026-10-15 09:13:37.019234", "current_ts": "2026-10-15T09:13:41.537004", "pos": "00000627270001984891", "primary_keys": ["ID"], "before": null, "after": { "ID": "1", "NAME": null, "EMPTY_VALUE": "", "LITERAL_NULL": "null" } } ### Expected behavior The parser should preserve the distinction between: - JSON null → Java null → SQL NULL - Empty string "" → empty string - Literal string "null" → literal string "null" ### Actual behavior The JSON null value in NAME is converted to the literal string "null". ### Root cause In AbstractJsonRecordParser.extractRowData(), the JSON record is converted to Map<String, Object>. JSON null is correctly represented as Java null at this stage. However, the subsequent conversion uses: Objects.toString(entry.getValue()) Objects.toString(null) returns the literal string "null". Simply changing this expression to Objects.toString(value, null) is insufficient because Collectors.toMap does not accept null values. ### Proposed fix Use a map collection approach that supports null values and explicitly preserve Java null. Keep the existing behavior for non-null basic values and JSON serialization of complex values. Add regression tests covering: - JSON null - Empty strings - The literal string "null" I have prepared a local fix and a parser regression test on the release-1.2.0 baseline. The regression test passes. ### Target branch My goal is to contribute this fix to release-1.2. Would a PR directly targeting release-1.2 be accepted, or should the fix be submitted to master first and then backported? ### Willingness to contribute I am willing to submit a PR with the fix and regression tests. ### Compute Engine Apache Flink 1.16.2 ### Minimal reproduce step 1. Use Apache Paimon 1.2.0 with Flink 1.16.2 and Java 8. 2. Create a Kafka topic named `ogg_null_test` and produce this OGG JSON message: ```json { "table": "TEST.CUSTOMER", "op_type": "I", "op_ts": "2026-10-15 09:13:37.019234", "current_ts": "2026-10-15T09:13:41.537004", "pos": "00000627270001984891", "primary_keys": ["ID"], "before": null, "after": { "ID": "1", "NAME": null, "EMPTY_VALUE": "", "LITERAL_NULL": "null" } } ``` 3. Run the Paimon `kafka_sync_database` action against this topic, using `value.format=ogg-json` and `scan.startup.mode=earliest-offset`. 4. Wait for a successful checkpoint, then query the resulting Paimon table using Flink SQL: ```sql SELECT ID, NAME IS NULL AS name_is_null, NAME = 'null' AS name_is_literal_null, EMPTY_VALUE = '' AS empty_value_preserved, LITERAL_NULL = 'null' AS literal_null_preserved FROM customer WHERE ID = '1'; ``` The table and column names may need to be adjusted to match the configured case conversion. Expected: - `name_is_null`: true - `name_is_literal_null`: NULL - `empty_value_preserved`: true - `literal_null_preserved`: true Actual: - `name_is_null`: false - `name_is_literal_null`: true The JSON null in `NAME` is converted to the literal string `"null"` by `AbstractJsonRecordParser.extractRowData()`, which calls `Objects.toString(value)`. ### What doesn't meet your expectations? I expect JSON null values in OGG messages to be preserved as SQL NULL when written to Paimon. However, with Paimon 1.2.0, a source field such as `"NAME": null` is written to a STRING column as the literal string "null". Therefore, `NAME IS NULL` returns false, while `NAME = 'null'` returns true. The parser should preserve the distinction between JSON null, an empty string "", and the literal string "null". ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
