ykisana opened a new pull request, #29628: URL: https://github.com/apache/flink/pull/29628
## What is the purpose of the change [FLINK-41086](https://issues.apache.org/jira/browse/FLINK-41086): `JSON_LENGTH(json, path)` and `JSON_TYPE(json, path)` should return NULL when the path does not locate a value, but `SqlJsonUtils#jsonLength` and `#jsonType` only caught `InvalidPathException` around `JsonPath#read`. JSON path functions throw other exceptions on values they cannot handle, and these escaped and failed the job: | Input | Path | Exception | |---|---|---| | `[]` | `$.min()` | `JsonPathException: Aggregation function attempted to calculate value using empty array` | | `{"a": []}` | `$.a.avg()` | same | | `{"a": [1, 2]}` | `$.a[*].min()` | same | | `[]` | `$.first()` | `IndexOutOfBoundsException` | | `{"a": [1, 2]}` | `$.a.index(2)` | `IndexOutOfBoundsException` | `JsonPathInputTypeStrategy` only rejects the `lax`/`strict` prefix, and whether a path function fails depends on the data, so these paths cannot be rejected at planning time. ## Brief change log - `SqlJsonUtils#jsonLength` and `#jsonType` catch `JsonPathException | IndexOutOfBoundsException` instead of `InvalidPathException` and return NULL. `JsonPathException` is the parent of `InvalidPathException` and `PathNotFoundException`, so inputs that returned NULL before still do. `IndexOutOfBoundsException` covers `first()`/`index(n)`, which do not throw a `JsonPathException`. Other exceptions still surface rather than being turned into NULL. - Updated the related comments. ## Verifying this change This change added tests and can be verified as follows: - Added cases to `JsonFunctionsITCase` (SQL and Table API) for `JSON_LENGTH` and `JSON_TYPE` over `min()`, `avg()`, `[*].min()`, `first()` and `index(n)` on values where they fail, expecting NULL. Inputs are table fields so the calls run at runtime rather than being constant-folded. - Without the fix, all 24 new test executions fail the job (15 with `JsonPathException`, 9 with `IndexOutOfBoundsException`); with it, `./mvnw -pl flink-table/flink-table-planner -Dtest=JsonFunctionsITCase test` passes (976 tests). - `./mvnw -pl flink-table/flink-table-runtime -Dtest=SqlJsonUtilsConversionTest test` passes. - `spotless:check` and `checkstyle:check` pass on both modules. Run locally with JDK 21 (`-Djdk21 -Pjava21-target`). ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): yes, but only the exception handling around `JsonPath#read` in `JSON_LENGTH`/`JSON_TYPE` changes; the success path is unchanged - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Opus 5.5 Generated by: Claude Opus 5.5 Fix written by and all code manually reviewed by @ykisana. Tests, PR description, and comments by Claude. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
