JingsongLi commented on code in PR #11236:
URL: https://github.com/apache/paimon/pull/11236#discussion_r4624004837
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/format/AbstractJsonRecordParser.java:
##########
@@ -86,29 +87,36 @@ protected Map<String, String> extractRowData(JsonNode
record, CdcSchema.Builder
fillDefaultTypes(record, schemaBuilder);
Map<String, Object> recordMap =
convertValue(record, new TypeReference<Map<String, Object>>()
{});
- Map<String, String> rowData =
- recordMap.entrySet().stream()
- .filter(entry -> Objects.nonNull(entry.getKey()))
- .collect(
- Collectors.toMap(
- Map.Entry::getKey,
- entry -> {
- if
(Objects.nonNull(entry.getValue())
- &&
!TypeUtils.isBasicType(entry.getValue())) {
- try {
- return
writeValueAsString(entry.getValue());
- } catch
(JsonProcessingException e) {
- LOG.error("Failed to
deserialize record.", e);
- return
Objects.toString(entry.getValue());
- }
- }
- return
Objects.toString(entry.getValue());
- }));
+ // A JSON null must stay null, so the value map has to accept null
values, which
+ // Collectors.toMap does not.
+ Map<String, String> rowData = new HashMap<>(recordMap.size());
+ recordMap.forEach(
+ (key, value) -> {
+ if (Objects.nonNull(key)) {
+ rowData.put(key, toRowDataValue(value));
+ }
+ });
evalComputedAndMetadataColumns(rowData, schemaBuilder);
return rowData;
}
+ @Nullable
+ private static String toRowDataValue(@Nullable Object value) {
+ if (Objects.isNull(value)) {
+ return null;
Review Comment:
[P2] Update the Maxwell and AWS DMS integration assertions to expect SQL NULL
Both `KafkaMaxwellSyncTableActionITCase.testFieldValNullSyncTable` and
`KafkaAWSDMSSyncTableActionITCase.testFieldValNullSyncTable` time out in
`CdcActionITCaseBase.waitForResult` in this PR's CI. They share
`KafkaSyncTableActionITCase.testTableFiledValNull`, whose expected row ends in
lowercase `null`. The fixture actually supplies JSON null, and
`DataFormatTestUtil.toStringNoRowKind` renders a real SQL NULL as uppercase
`NULL`. This intentional behavior change therefore leaves both tests polling
indefinitely for the old literal string. Please update the shared expected row
or assert `isNullAt` directly and rerun both IT cases. [Flink 1 CI
failure](https://github.com/apache/paimon/actions/runs/44223008664/job/134530298544)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]