Thanks for the explanation Ramin. LGTM
Cheers,
Timo
On 04.08.26 15:27, Ramin Gharib wrote:
Hello Timo,
Thanks for looking into this and the feedback.
Yes, for SQL: a Variant TIME lands as TIME(3), and per the FLIP a
sub-millisecond value fails the cast instead of truncating.
Small correction on the DataStream side: the ceiling for TIME is micros, not
nanos. Variant code 17 is TIME(isAdjustedToUTC=false, MICROS) and the spec has
no nanosecond TIME, so getTime() surfaces at most precision 6.
Nanosecond precision (9) applies to codes 18 and 19, the nanosecond timestamps,
not to TIME. TIMESTAMP(9) and TIMESTAMP_LTZ(9) already cover them, so both SQL
and DataStream keep the full nanoseconds. TIME is the only code with a gap,
since its Variant precision is micros (6) while SQL TIME caps at millis (3).
I've updated the FLIP to make this explicit (the TIME(3) mapping in the table
plus a note on the SQL-vs-flink-core precision).
Best,
Ramin
From: Timo Walther <[email protected]>
Date: Tuesday, 4. August 2026 at 12:59
To: [email protected] <[email protected]>
Subject: Re: [DISCUSS] FLIP-604: Complete VARIANT Primitive Coverage with UUID
and Timestamps
Hi Ramin,
thank you for evolving the VARIANT type further. I fully support the FLIP.
The ask for a UUID data type came up a couple of times in the past
because UUIDs are in almost every data pipeline.
Also full timestamp and time support incl nano second precision is a
topic that always pops up and support for it avoids friction.
One question the statement: "time without time zone (us)"
Since Flink SQL only supports TIME(3), I guess we limit VARIANT also to
millis within the SQL runtime, right? For DataStream API we can support
9, right?
Cheers,
Timo
On 04.08.26 12:28, Ramin Gharib wrote:
Hi everyone,
I'd like to start a discussion on FLIP-604: Complete VARIANT
Primitive Coverage with UUID and Timestamps. [1]
FLIP-521 [2] shipped VARIANT in Flink 2.1, but it only implements Parquet
Variant primitive codes 0 through 16. The open Variant spec goes
through 20: TIME, nanosecond timestamps, and UUID. So a VARIANT
written by Spark, Iceberg, or any Parquet writer that holds one of
these values cannot be read by Flink today, which breaks the
lakehouse round-trip VARIANT was meant to support.
This FLIP completes the primitive set. TIME and the nanosecond
timestamps reuse types Flink already has. UUID needs a type to land
on, so the FLIP also adds a native 16-byte UUID type as a first-class
SQL type, with casts, ordering, a UUID literal, and UUID_V4 and
UUID_V7 generator functions.
One point worth noting: no Calcite upgrade is required. Calcite has
had a UUID type since 1.39, the same release FLIP-521 adopted for
VARIANT, and Flink already runs Calcite 1.41. The 16-byte layout also
matches every engine with a native UUID type, including PostgreSQL,
Trino, DuckDB, ClickHouse, Snowflake, and Iceberg.
Looking forward to your feedback.
Best,
Ramin Gharib
[1]
https://cwiki.apache.org/confluence/spaces/FLINK/pages/446070993/FLIP-604+Complete+VARIANT+Primitive+Coverage+with+UUID+and+Timestamps
[2]
https://cwiki.apache.org/confluence/spaces/FLINK/pages/349637099/FLIP-521+Integrating+Variant+Type+into+Flink+Enabling+Efficient+Semi-Structured+Data+Processing
