This is an automated email from the ASF dual-hosted git repository.
raminqaf pushed a commit to branch release-2.4
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-2.4 by this push:
new 4ca6fb1c99c [BP-2.4][FLINK-41086][table] Return NULL from
`JSON_LENGTH` and `JSON_TYPE` when a JSON path function fails (#29805)
4ca6fb1c99c is described below
commit 4ca6fb1c99c171909352a0db08237bdd8dbdffd1
Author: Ramin Gharib <[email protected]>
AuthorDate: Fri Nov 6 15:23:20 2026 +0100
[BP-2.4][FLINK-41086][table] Return NULL from `JSON_LENGTH` and `JSON_TYPE`
when a JSON path function fails (#29805)
JSON_LENGTH(json, path) and JSON_TYPE(json, path) only caught
InvalidPathException around JsonPath#read. Path functions such as
min()/avg()/sum() throw JsonPathException on values they cannot handle,
e.g. an empty or non-numeric array. It escaped and failed the job
instead of returning NULL.
Catch JsonPathException, the parent of InvalidPathException and
PathNotFoundException. Other exceptions still surface.
Generated-by: Claude Opus 5.5
(cherry picked from commit 44131db0b9c9e01dc55ec7d2dae02a4c5fbe0e7e)
Co-authored-by: Yahya Kisana <[email protected]>
---
docs/data/sql_functions.yml | 6 ++--
docs/data/sql_functions_zh.yml | 6 ++--
.../planner/functions/JsonFunctionsITCase.java | 32 ++++++++++++++++++++++
.../table/runtime/functions/SqlJsonUtils.java | 13 +++++----
4 files changed, 48 insertions(+), 9 deletions(-)
diff --git a/docs/data/sql_functions.yml b/docs/data/sql_functions.yml
index c9aa68a6faa..696f6a09fe2 100644
--- a/docs/data/sql_functions.yml
+++ b/docs/data/sql_functions.yml
@@ -1294,9 +1294,10 @@ json:
description: |
Returns the number of elements in a JSON document, or the length of the
value at the specified path if one is provided.
- The input can be a JSON STRING or a VARIANT. Returns NULL if the
argument is NULL, the json is invalid, or the path is empty, malformed or does
not locate a value.
+ The input can be a JSON STRING or a VARIANT. Returns NULL if the
argument is NULL, the json is invalid, a path function fails (e.g., .min(),
.avg()), or the path is empty, malformed or does not locate a value.
The path must be a plain path literal such as '$.a.b'. A path carrying a
'lax'/'strict' path mode prefix raises an error.
eg.
+ ```sql
-- 2
JSON_LENGTH('{"1": "hello", "2": "bye bye"}')
@@ -1317,6 +1318,7 @@ json:
-- error: JSON_LENGTH does not support the 'lax'/'strict' path mode
prefix
JSON_LENGTH('{"1": "hello", "2": "bye bye"}', 'strict $.1')
+ ```
The length is determined as follows:
@@ -1337,7 +1339,7 @@ json:
table: jsonType(jsonValue[, path])
description: |
Returns a string value indicating the type of jsonValue, optionally at
the given JSON
- `path`. Returns SQL `NULL` if jsonValue is SQL `NULL`, is not valid
JSON, or the path
+ `path`. Returns SQL `NULL` if jsonValue is SQL `NULL`, is not valid
JSON, a path function fails (e.g., .min(), .avg()), or the path
does not resolve to exactly one value.
The possible results are `object`, `array`, `string`, `number`,
`boolean` and `null` for
diff --git a/docs/data/sql_functions_zh.yml b/docs/data/sql_functions_zh.yml
index 3d9de3802dd..9a90f434692 100644
--- a/docs/data/sql_functions_zh.yml
+++ b/docs/data/sql_functions_zh.yml
@@ -1387,9 +1387,10 @@ json:
description: |
Returns the number of elements in a JSON document, or the length of the
value at the specified path if one is provided.
- The input can be a JSON STRING or a VARIANT. Returns NULL if the
argument is NULL, the json is invalid, or the path is empty, malformed or does
not locate a value.
+ The input can be a JSON STRING or a VARIANT. Returns NULL if the
argument is NULL, the json is invalid, a path function fails (e.g., .min(),
.avg()), or the path is empty, malformed or does not locate a value.
The path must be a plain path literal such as '$.a.b'. A path carrying a
'lax'/'strict' path mode prefix raises an error.
eg.
+ ```sql
-- 2
JSON_LENGTH('{"1": "hello", "2": "bye bye"}')
@@ -1410,6 +1411,7 @@ json:
-- error: JSON_LENGTH does not support the 'lax'/'strict' path mode
prefix
JSON_LENGTH('{"1": "hello", "2": "bye bye"}', 'strict $.1')
+ ```
The length is determined as follows:
@@ -1430,7 +1432,7 @@ json:
table: jsonType(jsonValue[, path])
description: |
Returns a string value indicating the type of jsonValue, optionally at
the given JSON
- `path`. Returns SQL `NULL` if jsonValue is SQL `NULL`, is not valid
JSON, or the path
+ `path`. Returns SQL `NULL` if jsonValue is SQL `NULL`, is not valid
JSON, a path function fails (e.g., .min(), .avg()), or the path
does not resolve to exactly one value.
The possible results are `object`, `array`, `string`, `number`,
`boolean` and `null` for
diff --git
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/JsonFunctionsITCase.java
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/JsonFunctionsITCase.java
index aaeb24f6a0a..46d12c842b4 100644
---
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/JsonFunctionsITCase.java
+++
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/JsonFunctionsITCase.java
@@ -170,6 +170,23 @@ class JsonFunctionsITCase extends BuiltInFunctionTestBase {
$("f8").jsonLength(" "), "JSON_LENGTH(f8, ' ')",
null, INT().nullable())
.testResult($("f8").jsonLength(""), "JSON_LENGTH(f8, '')",
null, INT().nullable())
+ // a path function that fails on the value it reaches -> NULL
+ .testResult(
+ $("f10").jsonLength("$.min()"),
+ "JSON_LENGTH(f10, '$.min()')",
+ null,
+ INT().nullable())
+ .testResult(
+ $("f8").jsonLength("$.a.avg()"),
+ "JSON_LENGTH(f8, '$.a.avg()')",
+ null,
+ INT().nullable())
+ .testResult(
+ $("f8").jsonLength("$.a[*].min()"),
+ "JSON_LENGTH(f8, '$.a[*].min()')",
+ null,
+ INT().nullable())
+
// the root path on a scalar document behaves like the no-path
overload
.testResult($("f3").jsonLength("$"), "JSON_LENGTH(f3, '$')",
1, INT().nullable())
@@ -2024,6 +2041,21 @@ class JsonFunctionsITCase extends
BuiltInFunctionTestBase {
"number",
STRING().nullable()),
+ // A path function that fails on the value it reaches yields
NULL.
+ TestSetSpec.forFunction(BuiltInFunctionDefinitions.JSON_TYPE)
+ .onFieldsWithData("{\"a\": []}", "{\"a\": [1, 2]}")
+ .andDataTypes(STRING(), STRING())
+ .testResult(
+ $("f0").jsonType("$.a.avg()"),
+ "JSON_TYPE(f0, '$.a.avg()')",
+ null,
+ STRING().nullable())
+ .testResult(
+ $("f1").jsonType("$.a[*].min()"),
+ "JSON_TYPE(f1, '$.a[*].min()')",
+ null,
+ STRING().nullable()),
+
// The 'lax'/'strict' path mode prefix is rejected at planning
time, but a field of
// that name is addressed like any other.
TestSetSpec.forFunction(BuiltInFunctionDefinitions.JSON_TYPE)
diff --git
a/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/SqlJsonUtils.java
b/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/SqlJsonUtils.java
index 431d0248cf6..24c6a0d3558 100644
---
a/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/SqlJsonUtils.java
+++
b/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/SqlJsonUtils.java
@@ -39,6 +39,7 @@ import
org.apache.flink.shaded.com.jayway.jsonpath.Configuration;
import org.apache.flink.shaded.com.jayway.jsonpath.DocumentContext;
import org.apache.flink.shaded.com.jayway.jsonpath.InvalidPathException;
import org.apache.flink.shaded.com.jayway.jsonpath.JsonPath;
+import org.apache.flink.shaded.com.jayway.jsonpath.JsonPathException;
import org.apache.flink.shaded.com.jayway.jsonpath.Option;
import org.apache.flink.shaded.com.jayway.jsonpath.PathNotFoundException;
import org.apache.flink.shaded.com.jayway.jsonpath.spi.cache.CacheProvider;
@@ -437,7 +438,7 @@ public class SqlJsonUtils {
final String pathSpec,
final boolean isPathDefinite) {
// An empty path is ruled out up front because JsonPath rejects it
with an
- // IllegalArgumentException instead of the InvalidPathException caught
below.
+ // IllegalArgumentException instead of the JsonPathException caught
below.
if (parsedInput == null || parsedInput.hasException() ||
pathSpec.isEmpty()) {
return null;
}
@@ -450,8 +451,9 @@ public class SqlJsonUtils {
final Object value;
try {
value = JsonPath.parse(parsedInput.obj,
JSON_PATH_LENGTH_CONFIG).read(pathSpec);
- } catch (InvalidPathException e) {
- // The path does not exist, or is not a valid path at all.
+ } catch (JsonPathException e) {
+ // The path does not exist, is not a valid path at all, or applies
a path function
+ // that fails on the value it reaches, e.g. min() on an empty
array.
return null;
}
@@ -596,9 +598,10 @@ public class SqlJsonUtils {
final Object value;
try {
- // PathNotFoundException extends InvalidPathException (covers both
exceptions)
value = JsonPath.parse(parsedInput.obj,
JSON_PATH_TYPE_CONFIG).read(path);
- } catch (InvalidPathException e) {
+ } catch (JsonPathException e) {
+ // The path does not exist, is not a valid path at all, or applies
a path function
+ // that fails on the value it reaches, e.g. min() on an empty
array.
return null;
}
