[
https://issues.apache.org/jira/browse/FLINK-33656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789973#comment-17789973
]
duke commented on FLINK-33656:
------------------------------
If the type of the attentionList field is defined as Array, but the value does
not conform to the array parsing structure, and the attentionList field is at
the head or not at the end of the json message, all subsequent fields of
attentionList are returned as null.
For example: \{"attentionList":{},"id":"1234","sheetNo":"1700803757032"} In the
json string, attentionList is located at the head of the entire json message,
which is read in the flink sql json format. The source table structure is as
follows:
create table default_catalog.default_database.test (
id STRING
,sheetNo STRING
,attentionList ARRAY<STRING>
)
WITH ('connector' = 'kafka',
'topic' = 'test',
'properties.bootstrap.servers' = 'xxx',
'scan.startup.mode' = 'latest-offset',
'format' = 'json',
'json.ignore-parse-errors' = 'true',
'json.map-null-key.mode' = 'LITERAL'
);
Obviously the attentionList field is not parsed correctly, which will result in
the id and sheetNo fields being empty;
Suppose you modify the json message
:\{"id":"1234","attentionList":{},"sheetNo":"1700803757032"}, depending on the
location of attentionList, sheetNo will be empty.
> If json.ignose-parse-errors =true is configured and Array parsing errors
> occur, other columns will be empty
> -----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-33656
> URL: https://issues.apache.org/jira/browse/FLINK-33656
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.16.0, 1.17.0, 1.16.1, 1.16.2, 1.18.0, 1.17.1
> Reporter: duke
> Priority: Critical
> Attachments: image-2023-11-27-13-58-22-513.png,
> image-2023-11-27-13-59-42-066.png, image-2023-11-27-14-00-04-672.png,
> image-2023-11-27-14-00-41-176.png, image-2023-11-27-14-01-12-187.png,
> image-2023-11-27-14-02-01-252.png, image-2023-11-27-14-02-30-666.png,
> image-2023-11-27-14-02-52-065.png, image-2023-11-27-14-03-10-885.png
>
>
> If json.ignore-parse-errors is set to true and Array parsing errors occur,
> the fields following array are resolved as empty in the complete json message
> 1. !image-2023-11-27-13-59-42-066.png!
> !image-2023-11-27-14-00-04-672.png!
>
> 2. !image-2023-11-27-14-00-41-176.png!
> !image-2023-11-27-14-01-12-187.png!
> 3. !image-2023-11-27-14-02-52-065.png!
> !image-2023-11-27-14-03-10-885.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
