Jackie-Jiang commented on code in PR #16306:
URL: https://github.com/apache/pinot/pull/16306#discussion_r2196230684
##########
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java:
##########
@@ -321,17 +321,64 @@ private static void setValuesToMap(String keyColumnName,
String valueColumnName,
Map<String, String> objMap = (Map) obj;
result.put(objMap.get(keyColumnName), objMap.get(valueColumnName));
} else {
- ObjectMapper mapper = new ObjectMapper();
JsonNode mapNode;
try {
- mapNode = mapper.readTree(obj.toString());
+ mapNode = OBJECT_MAPPER.readTree(obj.toString());
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
result.put(mapNode.get(keyColumnName).asText(),
mapNode.get(valueColumnName).asText());
}
}
+ /**
+ * Returns all key paths in a JSON object up to the specified depth.
+ * Each key path is dot-separated for nested keys.
+ * @param jsonObj JSON object or string
+ * @param maxDepth Maximum depth to recurse (inclusive)
+ * @return Array of key paths (dot notation)
+ */
+ @ScalarFunction(nullableParameters = true)
+ public static List jsonKeys(@Nullable Object jsonObj, int maxDepth) {
Review Comment:
Why do we allow null json object? When any input is null, output should be
null, so we don't need to annotate it with `nullableParameters`
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonKeysTransformFunction.java:
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.scalar.JsonFunctions;
+import org.apache.pinot.core.operator.ColumnContext;
+import org.apache.pinot.core.operator.blocks.ValueBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+
+
+/**
+ * Transform function for jsonKeys(jsonField, depth)
+ * Returns all key paths in a JSON object up to the specified depth as a
multi-value string column.
+ */
+public class JsonKeysTransformFunction extends BaseTransformFunction {
+ public static final String FUNCTION_NAME = "jsonKeys";
+
+ private TransformFunction _jsonFieldTransformFunction;
+ private int _depth;
+
+ @Override
+ public String getName() {
+ return FUNCTION_NAME;
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String,
ColumnContext> columnContextMap) {
+ super.init(arguments, columnContextMap);
+ if (arguments.size() != 2) {
+ throw new IllegalArgumentException("jsonKeys expects 2 arguments:
jsonField, depth");
+ }
+ _jsonFieldTransformFunction = arguments.get(0);
+ TransformFunction depthArg = arguments.get(1);
Review Comment:
Are we going to take other arguments in the future? such as prefix?
If so, we should design it in a future proof way. Is there a function in
other DB that we can reference?
##########
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java:
##########
@@ -321,17 +321,64 @@ private static void setValuesToMap(String keyColumnName,
String valueColumnName,
Map<String, String> objMap = (Map) obj;
result.put(objMap.get(keyColumnName), objMap.get(valueColumnName));
} else {
- ObjectMapper mapper = new ObjectMapper();
JsonNode mapNode;
try {
- mapNode = mapper.readTree(obj.toString());
+ mapNode = OBJECT_MAPPER.readTree(obj.toString());
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
result.put(mapNode.get(keyColumnName).asText(),
mapNode.get(valueColumnName).asText());
}
}
+ /**
+ * Returns all key paths in a JSON object up to the specified depth.
+ * Each key path is dot-separated for nested keys.
+ * @param jsonObj JSON object or string
+ * @param maxDepth Maximum depth to recurse (inclusive)
+ * @return Array of key paths (dot notation)
+ */
+ @ScalarFunction(nullableParameters = true)
+ public static List jsonKeys(@Nullable Object jsonObj, int maxDepth) {
+ if (jsonObj == null || maxDepth < 1) {
+ return java.util.Collections.emptyList();
+ }
+ JsonNode root;
+ try {
+ if (jsonObj instanceof String) {
+ root = OBJECT_MAPPER.readTree((String) jsonObj);
+ } else if (jsonObj instanceof Map || jsonObj instanceof List) {
+ root = OBJECT_MAPPER.valueToTree(jsonObj);
+ } else if (jsonObj instanceof JsonNode) {
+ root = (JsonNode) jsonObj;
+ } else {
+ root = OBJECT_MAPPER.readTree(jsonObj.toString());
+ }
+ } catch (Exception e) {
+ return java.util.Collections.emptyList();
+ }
+ List<String> keys = new java.util.ArrayList<>();
+ collectKeysForJsonKeys(root, "", 1, maxDepth, keys);
+ return keys;
+ }
+
+ private static void collectKeysForJsonKeys(JsonNode node, String prefix, int
depth, int maxDepth, List<String> keys) {
+ if (node == null || depth > maxDepth) {
Review Comment:
Can `node` ever be null?
##########
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java:
##########
@@ -321,17 +321,64 @@ private static void setValuesToMap(String keyColumnName,
String valueColumnName,
Map<String, String> objMap = (Map) obj;
result.put(objMap.get(keyColumnName), objMap.get(valueColumnName));
} else {
- ObjectMapper mapper = new ObjectMapper();
JsonNode mapNode;
try {
- mapNode = mapper.readTree(obj.toString());
+ mapNode = OBJECT_MAPPER.readTree(obj.toString());
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
result.put(mapNode.get(keyColumnName).asText(),
mapNode.get(valueColumnName).asText());
}
}
+ /**
+ * Returns all key paths in a JSON object up to the specified depth.
+ * Each key path is dot-separated for nested keys.
+ * @param jsonObj JSON object or string
+ * @param maxDepth Maximum depth to recurse (inclusive)
+ * @return Array of key paths (dot notation)
+ */
+ @ScalarFunction(nullableParameters = true)
+ public static List jsonKeys(@Nullable Object jsonObj, int maxDepth) {
+ if (jsonObj == null || maxDepth < 1) {
Review Comment:
When `maxDepth` is non-positive, do we want to treat it as unlimited?
##########
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java:
##########
@@ -252,10 +253,9 @@ public static Object jsonKeyValueArrayToMap(Object
keyValueArray, String keyColu
setValuesToMap(keyColumnName, valueColumnName, obj, result);
}
} else {
- ObjectMapper mapper = new ObjectMapper();
JsonNode arrayNode;
try {
- arrayNode = mapper.readTree(keyValueArray.toString());
+ arrayNode = OBJECT_MAPPER.readTree(keyValueArray.toString());
Review Comment:
Not introduced in this PR, but you might want to use `JsonUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
