[
https://issues.apache.org/jira/browse/FLINK-41298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuepeng Pan updated FLINK-41298:
--------------------------------
Fix Version/s: 2.5.0
> Document and test that JSON input is read only up to its first value
> --------------------------------------------------------------------
>
> Key: FLINK-41298
> URL: https://issues.apache.org/jira/browse/FLINK-41298
> Project: Flink
> Issue Type: Improvement
> Components: API / Core, Table SQL / Runtime
> Affects Versions: 2.1.0, 2.2.0, 2.3.0
> Reporter: Ramin Gharib
> Assignee: Ramin Gharib
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.5.0
>
>
> {{PARSE_JSON}}, {{TRY_PARSE_JSON}}, the {{raw}} format's VARIANT column and
> the JSON functions read the first JSON value of their input and ignore what
> follows. This is the intended behavior, but no doc states it and no test pins
> it.
> {noformat}
> SELECT JSON_STRING(PARSE_JSON('{"a":1} xyz')); -- {"a":1}
> SELECT JSON_STRING(PARSE_JSON('{"a":1}{"a":2}')); -- {"a":1}
> SELECT JSON_STRING(TRY_PARSE_JSON('{"a":1} xyz')); -- {"a":1}
> SELECT '{"a":1} xyz' IS JSON; -- TRUE
> {noformat}
> A {{raw}} message that holds newline-delimited JSON without
> {{raw.line-delimiter}} keeps only its first document.
> *Decision*
> Keep the behavior. Spark's {{parse_json}} does the same, and lenient parsing
> is Jackson's default. Rejecting the content would turn input that parses
> today into errors in running jobs. FLINK-40225 made the same call for {{IS
> JSON}}, {{JSON_VALUE}} and the other JSON functions.
> *Scope*
> # A test in {{BinaryVariantInternalBuilderTest}} that pins the behavior for
> {{PARSE_JSON}}, {{TRY_PARSE_JSON}} and the {{raw}} format, which share
> {{BinaryVariantInternalBuilder.parseJson}}.
> # An {{IS JSON}} case in {{JsonFunctionsITCase}} that pins it for the parser
> the JSON functions share.
> # A one-line note in the docs of {{PARSE_JSON}}, {{TRY_PARSE_JSON}}, the JSON
> functions and the {{raw}} format.
> This ticket first proposed rejecting the content after the first value. That
> proposal was dropped in favor of the scope above.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
