cloud-fan commented on code in PR #57479:
URL: https://github.com/apache/spark/pull/57479#discussion_r3737201322
##########
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/json/JsonExpressionUtils.java:
##########
@@ -77,4 +77,32 @@ public static GenericArrayData jsonObjectKeys(UTF8String
json) {
return null;
}
}
+
+ public static UTF8String jsonTypeof(UTF8String json) {
+ try (JsonParser jsonParser =
+ CreateJacksonParser.utf8String(SharedFactory.jsonFactory(), json)) {
+ JsonToken token = jsonParser.nextToken();
+ if (token == null) {
+ return null;
+ }
+ String type = switch (token) {
+ case START_OBJECT -> "object";
+ case START_ARRAY -> "array";
+ case VALUE_STRING -> "string";
+ case VALUE_NUMBER_INT, VALUE_NUMBER_FLOAT -> "number";
+ case VALUE_TRUE, VALUE_FALSE -> "boolean";
+ case VALUE_NULL -> "null";
+ default -> null;
+ };
+ if (type == null) {
+ return null;
+ }
+ // Consume the value so malformed input surfaces as a parse error and
returns null,
+ // matching json_object_keys and json_array_length.
+ jsonParser.skipChildren();
Review Comment:
Please verify that the parser reached end-of-input before returning the
type. An input containing two JSON values such as `123 true` is not a valid
single JSON document, but this path returns `number` instead of the documented
null; call `nextToken()` after consuming the value, require EOF, and add a
trailing-content test.
##########
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/json/JsonExpressionUtils.java:
##########
@@ -77,4 +77,32 @@ public static GenericArrayData jsonObjectKeys(UTF8String
json) {
return null;
}
}
+
+ public static UTF8String jsonTypeof(UTF8String json) {
+ try (JsonParser jsonParser =
+ CreateJacksonParser.utf8String(SharedFactory.jsonFactory(), json)) {
+ JsonToken token = jsonParser.nextToken();
+ if (token == null) {
+ return null;
+ }
+ String type = switch (token) {
+ case START_OBJECT -> "object";
+ case START_ARRAY -> "array";
+ case VALUE_STRING -> "string";
+ case VALUE_NUMBER_INT, VALUE_NUMBER_FLOAT -> "number";
+ case VALUE_TRUE, VALUE_FALSE -> "boolean";
+ case VALUE_NULL -> "null";
+ default -> null;
+ };
+ if (type == null) {
+ return null;
+ }
+ // Consume the value so malformed input surfaces as a parse error and
returns null,
+ // matching json_object_keys and json_array_length.
+ jsonParser.skipChildren();
+ return UTF8String.fromString(type);
Review Comment:
Please cache the six possible `UTF8String` results and return those
constants from the switch. This currently re-encodes a fixed literal and
allocates a byte array plus wrapper for every evaluated row.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
