ganeshashree opened a new pull request, #58005:
URL: https://github.com/apache/spark/pull/58005
### What changes were proposed in this pull request?
Implement the ANSI SQL:2016 `JSON_ARRAY` constructor function (feature
T811), which builds a JSON array from a list of argument values and returns it
as JSON text.
Syntax (a dedicated `primaryExpression` production, mirroring `JSON_TABLE`,
because the
argument list carries non-expression syntax):
JSON_ARRAY ( [ value [ FORMAT JSON ] [, ...] ]
[ { NULL | ABSENT } ON NULL ]
[ RETURNING data_type ] )
- New `JsonArray` Catalyst expression that serializes each element through
the same
Jackson writer as `to_json`, so numbers, decimals, datetimes, and nested
structs/arrays/maps render consistently.
- `(NULL | ABSENT) ON NULL`: `ABSENT ON NULL` (the standard default) drops
NULL
elements; `NULL ON NULL` keeps them as JSON `null`.
- `RETURNING`: defaults to `STRING` and is restricted to string types
(VARIANT deferred);
`CHAR`/`VARCHAR` are normalized to `STRING`.
- Per-element `FORMAT JSON`: splices already-JSON string text verbatim
instead of quoting
it (validated for well-formedness at eval). A nested JSON constructor
carries it
implicitly. The splice/quote decision is frozen at parse time so a later
optimizer
rewrite (e.g. `CollapseProject` inlining a `JSON_ARRAY` alias) cannot flip
it.
- `JSON_ARRAY` and `ABSENT` added as non-reserved keywords (matching
`JSON_TABLE`/`JSON_VALUE`).
Example:
SELECT JSON_ARRAY(1, 'x', true); -- [1,"x",true]
SELECT JSON_ARRAY(1, NULL, 3); -- [1,3] (ABSENT ON
NULL default)
SELECT JSON_ARRAY(1, NULL, 3 NULL ON NULL); -- [1,null,3]
SELECT JSON_ARRAY(JSON_ARRAY(1, 2), 3); -- [[1,2],3] (nested:
implicit FORMAT JSON)
SELECT JSON_ARRAY('[1,2]'); -- ["[1,2]"] (plain
string: quoted)
SELECT JSON_ARRAY('[1,2]' FORMAT JSON); -- [[1,2]] (explicit
FORMAT JSON: spliced)
### Why are the changes needed?
Spark implements none of the ANSI SQL:2016 constructor JSON functions,
forcing users migrating from Oracle, SQL Server, MySQL, Trino, or BigQuery to
rewrite `JSON_ARRAY(a, b, c)` as `to_json(array(a, b, c))`, which is verbose
and does not carry the standard's `ON NULL` / `RETURNING` / `FORMAT JSON`
semantics.
### Does this PR introduce _any_ user-facing change?
Yes. It adds the new `JSON_ARRAY` SQL constructor function and the new
non-reserved keywords `JSON_ARRAY` and `ABSENT`. There is no change to existing
queries (both keywords remain usable as identifiers, as they are non-reserved).
### How was this patch tested?
- New `JsonArraySuite` (49 cases): scalar/mixed/nested values, both `ON
NULL` modes, `RETURNING` (collated + CHAR/VARCHAR normalization),
implicit/explicit `FORMAT JSON` (splicing, per-row validation, error
truncation), plan-stability of the splice decision,
foldable/throwable/collation behavior, and analysis-time type rejection.
- Golden-file coverage in `json-functions.sql` (`SQLQueryTestSuite`).
- `SQLKeywordSuite` and regenerated `keywords*.sql.out` for the new
keywords; `checkError` coverage for the new error conditions.
- New reference doc page `sql-ref-syntax-qry-select-json-array.md`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
