Hi everyone,
After a long discussion with Timo Walther, I updated the FLIP to
change how the functions handle SQL NULL.
The functions now handle SQL NULL and variant null consistently. A
SQL NULL argument gives a SQL NULL result.
To write a variant null on purpose, the FLIP adds two functions:
- VARIANT_NULL() returns a variant null, the same as
PARSE_JSON('null').
- TO_VARIANT(x) is the same as CAST(x AS VARIANT), except for SQL
NULL. CAST(NULL AS VARIANT) returns NULL, so TO_VARIANT gives a
variant null instead.
For example, with discount = NULL:
VARIANT_SET(payload, '$.discount', discount)
-- NULL
VARIANT_SET(payload, '$.discount', TO_VARIANT(discount))
-- {"discount":null,"order_id":8,"user":"bob"}
Other changes:
- VARIANT_OBJECT, VARIANT_ARRAY and VARIANT_MERGE return NULL for a
SQL NULL argument. A NULL key in VARIANT_OBJECT is still an error.
- VARIANT_OBJECT_AGG skips rows with a SQL NULL value.
- VARIANT_DELETE and VARIANT_PICK still skip a NULL path.
- The prior-art table has a new column that names the proposed
function for each row.
One point is still open. IS_VARIANT_NULL now returns NULL for SQL
NULL. Returning FALSE instead would keep its result NOT NULL. The
wiki has an example under Open Questions. I'm happy to hear your
opinion on it.
Wiki:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-618%3A+Edit%2C+Inspect+and+Build+VARIANT+Values
Cheers,
Ramin Gharib
On Thu, Oct 22, 2026 at 5:36 PM Ramin Gharib <[email protected]> wrote:
> Hi everyone,
>
> I'd like to start a discussion on FLIP-618: Edit, Inspect and Build
> VARIANT Values.
>
> Wiki:
> https://cwiki.apache.org/confluence/spaces/FLINK/pages/461570450/FLIP-618+Edit+Inspect+and+Build+VARIANT+Values
>
> Flink can store and read VARIANT values since 2.1. SQL users still
> cannot inspect or change one field and keep the rest, ask what a value
> contains,
> or build an object from columns without going through JSON text.
> Today each of these needs a UDF that rebuilds the whole value.
>
> The FLIP proposes built-in functions to edit (VARIANT_SET,
> VARIANT_DELETE, VARIANT_MERGE, ...), inspect (IS_VARIANT_NULL,
> VARIANT_KEYS, SCHEMA_OF_VARIANT, ...) and build (VARIANT_OBJECT,
> VARIANT_OBJECT_AGG) VARIANT values, plus a Java API to edit a value
> inside a UDF.
>
> Looking forward to your feedback.
>
> Best,
> Ramin Gharib
>
