Hi Fabian thanks for starting the discussion
+1 to align the behavior with standard and other vendors On Fri, Jul 10, 2026 at 12:15 PM Timo Walther <[email protected]> wrote: > > Hi Fabian, > > it is very common in the SQL area that passing a NULL as input to a > function results in NULL as output. I would consider this as a bug that > we should fix asap. From my gut feeling, I would say that IS JSON > functions are not very popular. If we were talking about JSON_VALUE or > JSON_QUERY, things would be different. So +1 for the breaking change. > > Cheers, > Timo > > > On 09.07.26 14:24, Fabian Hueske wrote: > > Hi folks, > > > > I'd like to discuss FLINK-39943 [1]. > > Today, Flink's IS JSON / IS NOT JSON functions are hard-coded to never > > return NULL. > > The recent Calcite 1.38.0 upgrade (FLINK-36602) changed Calcite's own > > default to the standard-compliant behavior instead: IS JSON is a predicate > > subject to three-valued logic and should evaluate to UNKNOWN (NULL) when > > the input is NULL, not FALSE. > > Flink currently suppresses this new Calcite default to preserve the old, > > standards-incompatible semantics [2]. > > I've checked a few other engines and all agree with the standard here. > > Oracle, PostgreSQL (16+), SQL Server (ISJSON), MySQL (JSON_VALID), and > > Snowflake (IS_OBJECT/IS_ARRAY/CHECK_JSON) all return NULL/UNKNOWN for a > > NULL input. > > > > Originally, Flink implemented the correct nullable behavior when the > > functions were added in Flink 1.11. For Flink 1.15, this was hotfixed [3] > > to comply with Calcite's non-null semantics. > > Since Calcite fixed its definition with the 1.38 release, I'm proposing > > that we revert the Flink 1.15 hotfix, even though it's a breaking change. > > > > * PRO: standards compliance, consistency with every other engine and with > > Calcite's own default (less surprise porting SQL from other systems), > > consistency with Flink's other JSON functions which already propagate NULL, > > and removing custom codegen that fights Calcite on every future upgrade. > > * CON: it's a breaking change, most notably flipping NULL IS NOT JSON from > > TRUE to NULL, which can affect existing filters, views, or NOT NULL-typed > > computed columns/sinks built on this predicate, and there's no > > compatibility flag today to ease the transition. > > > > This is one of several breaking side-effects of the Calcite 1.38.0 > > upgrade. Hence, I'd propose treating it as a deliberate, documented break > > targeted at the next minor version (2.4) rather than adding a dedicated > > config flag. > > > > Curious to hear if others agree, or see a strong case for keeping the > > current NOT NULL semantics. > > > > Best, > > Fabian > > > > [1] https://issues.apache.org/jira/browse/FLINK-39943 > > [2] > > https://github.com/apache/flink/commit/8de3293e2d#diff-53c19443d4ddaabe90d0041379f3656f3d52983ae7a0226a9b23bc87675471a8R93 > > [3] > > https://github.com/apache/flink/commit/74146c626edd7603f00a6f0c90c7fccaf2066912 > > > -- Best regards, Sergey
