JingsongLi commented on code in PR #11236:
URL: https://github.com/apache/paimon/pull/11236#discussion_r4624004837


##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/format/AbstractJsonRecordParser.java:
##########
@@ -86,29 +87,36 @@ protected Map<String, String> extractRowData(JsonNode 
record, CdcSchema.Builder
         fillDefaultTypes(record, schemaBuilder);
         Map<String, Object> recordMap =
                 convertValue(record, new TypeReference<Map<String, Object>>() 
{});
-        Map<String, String> rowData =
-                recordMap.entrySet().stream()
-                        .filter(entry -> Objects.nonNull(entry.getKey()))
-                        .collect(
-                                Collectors.toMap(
-                                        Map.Entry::getKey,
-                                        entry -> {
-                                            if 
(Objects.nonNull(entry.getValue())
-                                                    && 
!TypeUtils.isBasicType(entry.getValue())) {
-                                                try {
-                                                    return 
writeValueAsString(entry.getValue());
-                                                } catch 
(JsonProcessingException e) {
-                                                    LOG.error("Failed to 
deserialize record.", e);
-                                                    return 
Objects.toString(entry.getValue());
-                                                }
-                                            }
-                                            return 
Objects.toString(entry.getValue());
-                                        }));
+        // A JSON null must stay null, so the value map has to accept null 
values, which
+        // Collectors.toMap does not.
+        Map<String, String> rowData = new HashMap<>(recordMap.size());
+        recordMap.forEach(
+                (key, value) -> {
+                    if (Objects.nonNull(key)) {
+                        rowData.put(key, toRowDataValue(value));
+                    }
+                });
         evalComputedAndMetadataColumns(rowData, schemaBuilder);
 
         return rowData;
     }
 
+    @Nullable
+    private static String toRowDataValue(@Nullable Object value) {
+        if (Objects.isNull(value)) {
+            return null;

Review Comment:
   [P2] Update the Maxwell and AWS DMS integration assertions to expect SQL NULL
   
   Both `KafkaMaxwellSyncTableActionITCase.testFieldValNullSyncTable` and 
`KafkaAWSDMSSyncTableActionITCase.testFieldValNullSyncTable` time out in 
`CdcActionITCaseBase.waitForResult` in this PR's CI. They share 
`KafkaSyncTableActionITCase.testTableFiledValNull`, whose expected row ends in 
lowercase `null`. The fixture actually supplies JSON null, and 
`DataFormatTestUtil.toStringNoRowKind` renders a real SQL NULL as uppercase 
`NULL`. This intentional behavior change therefore leaves both tests polling 
indefinitely for the old literal string. Please update the shared expected row 
or assert `isNullAt` directly and rerun both IT cases. [Flink 1 CI 
failure](https://github.com/apache/paimon/actions/runs/44223008664/job/134530298544)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to