[
https://issues.apache.org/jira/browse/FLINK-40828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40828:
-----------------------------------
Labels: pull-request-available (was: )
> Make VARIANT toString() and result printing never fail
> ------------------------------------------------------
>
> Key: FLINK-40828
> URL: https://issues.apache.org/jira/browse/FLINK-40828
> Project: Flink
> Issue Type: Improvement
> Components: API / Core, Table SQL / Planner, Table SQL / Runtime
> Reporter: Ramin Gharib
> Assignee: Ramin Gharib
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> {\{Variant#toString()}} delegates to \{{toJson()}}, which throws for values
> JSON cannot represent. A VARIANT can hold such values, for example NaN from
> Avro data or a UDF. It can also hold a type id written by a newer version.
> Such a VARIANT breaks logs, test failure messages, and printed query results.
> ||Case||Current behavior||
> |\{{toString()}} of \{{[1, NaN]}}|throws \{{Non-finite value NaN cannot be
> serialized to JSON.}}|
> |Print a VARIANT column holding \{{[1, NaN]}}|throws \{{Error when casting
> VARIANT to STRING.}}|
> |Print a VARIANT column holding a JSON \{{null}}|prints an empty cell|
> |\{{toString()}} or print a VARIANT with an unknown type id|throws
> \{{UNKNOWN_PRIMITIVE_TYPE_IN_VARIANT}}, the whole value is lost|
> h3. Proposal
> * Add an internal \{{VariantFormatter}} interface: \{{String
> format(Variant)}}.
> * Move the JSON rendering out of \{{BinaryVariant}} into
> \{{JsonVariantFormatter}}:
> ** \{{STRICT}} backs \{{toJson()}}. Output is unchanged, and it still fails
> on NaN, infinity, and undecodable data, so \{{toJson()}} always returns valid
> JSON.
> ** \{{LENIENT}} backs \{{toString()}}. It never fails and is meant for
> debugging. NaN becomes \{{"NaN"}}. An unknown type id becomes
> \{{"<UNKNOWN>"}}, other undecodable data \{{"<INVALID>"}}. The rest of the
> value still renders.
> * Print VARIANT results through \{{SqlStringVariantFormatter}}. It renders
> like \{{CAST(v AS STRING)}}, the same way MAP and ARRAY print, and never
> fails.
> ||Stored||Printed today||Printed after||
> |\{{{"a":1,"b":["x",null]}}}|\{{{"a":1,"b":["x",null]}}}|\{{{a=1, b=[x,
> NULL]}}}|
> |\{{[1, NaN]}}|fails|\{{[1, NaN]}}|
> |JSON \{{null}}|empty cell|\{{NULL}}|
> |bytes \{{0xC328}}|\{{"wyg="}}|\{{x'c328'}}|
> |\{{[1, <type id 31>]}}|fails|\{{[1, <UNKNOWN>]}}|
> h3. Compatibility
> * No public API change. The new classes are \{{@Internal}}. The javadoc of
> \{{Variant#toJson()}} documents the contract.
> * \{{toJson()}}, \{{JSON_STRING}}, the json and raw formats, and \{{CAST(v AS
> STRING)}} are unchanged.
> * Printed VARIANT output changes from JSON to the CAST form, so strings are
> no longer quoted.
> * No state or serialization change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
