[
https://issues.apache.org/jira/browse/FLINK-33656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17791482#comment-17791482
]
duke edited comment on FLINK-33656 at 11/30/23 7:05 AM:
--------------------------------------------------------
Hi,[~martijnvisser]
I found that in version 1.18 json parsing uses a new class class, and the
method is controlled by the decode.json-parser.enabled parameter, but this
parameter does not take effect according to the value configured in the sql.
And array resolution through JsonParserRowDataDeserializationSchema class
should be using the reconstruction after the class, resulting in abnormal array
resolution after eventually lead to the problem.
!image-2023-11-30-14-47-02-300.png|width=353,height=321!
was (Author: JIRAUSER303286):
Hi,[~martijnvisser]
I found that in version 1.18 json parsing uses a new class class, and the
method is controlled by the decode. Json-parser. enabled parameter, but this
parameter does not take effect according to the value configured in the sql.
And array resolution through JsonParserRowDataDeserializationSchema class
should be using the reconstruction after the class, resulting in abnormal array
resolution after eventually lead to the problem.
!image-2023-11-30-14-47-02-300.png|width=353,height=321!
> If json.ignose-parse-errors =true is configured and Array parsing errors
> occur, other columns will be empty
> -----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-33656
> URL: https://issues.apache.org/jira/browse/FLINK-33656
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.16.0, 1.17.0, 1.16.1, 1.16.2, 1.18.0, 1.17.1
> Reporter: duke
> Priority: Critical
> Attachments: image-2023-11-27-13-58-22-513.png,
> image-2023-11-27-13-59-42-066.png, image-2023-11-27-14-00-04-672.png,
> image-2023-11-27-14-00-41-176.png, image-2023-11-27-14-01-12-187.png,
> image-2023-11-27-14-02-01-252.png, image-2023-11-27-14-02-30-666.png,
> image-2023-11-27-14-02-52-065.png, image-2023-11-27-14-03-10-885.png,
> image-2023-11-30-14-47-02-300.png
>
>
> If json.ignore-parse-errors is set to true and Array parsing errors occur,
> the fields following array are resolved as empty in the complete json message
> Create Table DDL
> create table default_catalog.default_database.test (
> id STRING
> ,sheetNo STRING
> ,attentionList ARRAY<STRING>
> )
> WITH ('connector' = 'kafka',
> 'topic' = 'test',
> 'properties.bootstrap.servers' = 'xxx',
> 'scan.startup.mode' = 'latest-offset',
> 'format' = 'json',
> 'json.ignore-parse-errors' = 'true',
> 'json.map-null-key.mode' = 'LITERAL'
> );
> 1.、 !image-2023-11-27-13-59-42-066.png|width=501,height=128!
> !image-2023-11-27-14-00-04-672.png|width=631,height=85!
> 2.
> !image-2023-11-27-14-00-41-176.png|width=506,height=67!
> !image-2023-11-27-14-01-12-187.png|width=493,height=118!
> 3.
> !image-2023-11-27-14-02-52-065.png|width=493,height=181!
> !image-2023-11-27-14-03-10-885.png|width=546,height=144!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
