vinodkc commented on code in PR #58024:
URL: https://github.com/apache/spark/pull/58024#discussion_r4474115088
##########
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/json/JsonExpressionUtils.java:
##########
@@ -117,4 +122,151 @@ public static UTF8String jsonTypeof(UTF8String json) {
return null;
}
}
+
+ // Shape codes for the SQL `IS [NOT] JSON` predicate, kept in sync with
IsJsonShape on the
+ // Scala side. SHAPE_ANY covers both bare `IS JSON` and `IS JSON VALUE` (any
well-formed value).
+ public static final int SHAPE_ANY = 0;
+ public static final int SHAPE_OBJECT = 1;
+ public static final int SHAPE_ARRAY = 2;
+ public static final int SHAPE_SCALAR = 3;
+
+ // A strict JSON factory for the ANSI SQL `IS [NOT] JSON` predicate. Unlike
the shared factory
+ // used by the Hive-compatible JSON functions, it leaves ALLOW_SINGLE_QUOTES
and
+ // ALLOW_UNESCAPED_CONTROL_CHARS disabled, so only well-formed ANSI JSON is
accepted
+ // (e.g. `{'a':1}` with single quotes is rejected).
+ private static final JsonFactory STRICT_JSON_FACTORY = new JsonFactory();
+
+ /**
+ * Evaluates the SQL `IS JSON` predicate for a single, already-evaluated
operand: returns whether
+ * {@code json} is a well-formed ANSI JSON value of the requested {@code
shape}. The operand is
+ * read exactly once here, so a non-deterministic input is evaluated a
single time (unlike a
+ * plan that references the child in more than one place). NULL handling
(NULL input yields a
+ * NULL result) is left to the calling expression; this method is only
invoked for non-null
+ * input. Malformed input yields false. Input that exceeds one of Jackson's
default
+ * {@code StreamReadConstraints} (nesting depth, number length, string
length, or field name
+ * length) cannot be classified and raises an IS_JSON_INPUT_LIMIT_EXCEEDED
condition, at the
+ * same limits where {@code parse_json} fails -- but only if checking the
input actually
+ * requires going past a limit: a shape mismatch decided by the first token
(including an
+ * over-long root number under OBJECT or ARRAY, which cannot be an object or
an array)
+ * returns false, like any other mismatch.
+ */
+ public static boolean isJson(UTF8String json, int shape) {
+ // The parser decodes the bytes through a UTF-8 reader that silently
replaces malformed
+ // sequences with U+FFFD, so check the raw bytes first: input that is not
valid UTF-8
+ // (e.g. a lone continuation byte, or an encoded surrogate) is not
well-formed JSON text.
+ if (!json.isValid()) {
+ return false;
+ }
+ try (JsonParser jsonParser =
+ CreateJacksonParser.utf8String(STRICT_JSON_FACTORY, json)) {
+ JsonToken token;
+ try {
+ token = jsonParser.nextToken();
+ } catch (StreamConstraintsException e) {
+ // Jackson validates a number's length while producing its token, and
no other limit can
+ // be reached while producing the first token (nesting needs a second
token, and strings
Review Comment:
Fixed.
The comment now gives the correct reason: nesting needs a second token,
string values are decoded lazily at getText(), and the first token is never a
field name (a field name only appears inside an object, after START_OBJECT).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
