ganeshashree opened a new pull request, #57920:
URL: https://github.com/apache/spark/pull/57920
**What changes were proposed in this pull request?**
Add the ANSI SQL:2016 JSON_EXISTS predicate (feature T821):
JSON_EXISTS(jsonExpr, path [ { TRUE | FALSE | UNKNOWN | ERROR } ON ERROR ])
It returns whether a SQL/JSON path matches at least one item in a JSON
document (BOOLEAN):
- a match (including a match whose value is JSON null) → true; no match →
false;
- a SQL NULL input → NULL (Unknown, per 9075-2 8.23);
- input that is not a single well-formed JSON value → the ON ERROR clause
(default FALSE).
Unlike get_json_object(...) IS NOT NULL, this distinguishes "present but
null" from "absent". Paths are evaluated in lax mode, matching Oracle and
PostgreSQL 17: wildcards ([*], .*, embedded) are supported, arrays are
auto-wrapped/unwrapped, and a structural mismatch is a non-match rather than an
error. JSON_EXISTS is added as a non-reserved keyword with a dedicated grammar
production (parser-only, like JSON_TABLE), and evaluation reuses JSON_TABLE's
token-aware path evaluator via a new single-pass pathExists/anyMatch.
**Why are the changes needed?**
Spark implements none of the ANSI scalar JSON functions. JSON_EXISTS is
broadly supported (Oracle, PostgreSQL, SQL Server, DB2), so its absence forces
migrated queries to approximate it with get_json_object(...) IS NOT NULL, which
is string-only and conflates a present-null value with an absent path.
**Does this PR introduce any user-facing change?**
Yes. It adds the new JSON_EXISTS SQL predicate and a SQL reference page. The
new JSON_EXISTS keyword is non-reserved in both ANSI and default modes, so
existing identifiers named json_exists continue to work.
**How was this patch tested?**
New JsonExistsSuite and json-functions.sql golden files (path
present/absent, present-null vs absent, lax wildcards with auto-wrap/unwrap,
out-of-range index, malformed → each ON ERROR behavior, trailing garbage, NULL
input, invalid-path rejection). Regenerated keyword golden files and updated
the getSQLKeywords assertions; existing ExpressionParserSuite, SQLKeywordSuite,
SparkThrowableSuite, SparkConnectDatabaseMetaDataSuite, and
ThriftServerWithSparkContextSuite pass.
**Was this patch authored or co-authored using generative AI tooling?**
Generated-by: Claude Code (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
