chenhao-db commented on code in PR #46011:
URL: https://github.com/apache/spark/pull/46011#discussion_r1561844116
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtils.scala:
##########
@@ -41,4 +41,15 @@ object VariantExpressionEvalUtils {
input.toString, BadRecordException(() => input, cause = e))
}
}
+
+ def isVariantNull(input: VariantVal): Boolean = {
+ if (input == null) {
+ // This is a SQL NULL, not a Variant NULL
+ false
+ } else {
+ val valMeta = input.getValue
Review Comment:
I think `valMeta` is a confusing name. You are really just checking the
`value`.
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtilsSuite.scala:
##########
@@ -122,4 +122,42 @@ class VariantExpressionEvalUtilsSuite extends
SparkFunSuite {
Map("sizeLimit" -> "16.0 MiB", "functionName" -> "`parse_json`"))
}
}
+
+ test("isVariantNull") {
Review Comment:
Recommend to also add SQL tests in `VariantEndToEndSuite`.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala:
##########
@@ -822,6 +822,7 @@ object FunctionRegistry {
// Variant
expression[ParseJson]("parse_json"),
+ expression[IsVariantNull]("is_variant_null"),
Review Comment:
I guess you need to look at `ExpressionsSchemaSuite`. It will fail if you
only register here but haven't modified `sql-expression-schema.md`.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/variantExpressions.scala:
##########
@@ -73,6 +73,45 @@ case class ParseJson(child: Expression)
copy(child = newChild)
}
+@ExpressionDescription(
+ usage = "_FUNC_(expr) - Check if a variant value is a variant NULL.",
Review Comment:
We'd better be explicit in the doc: it returns true if and only if the input
is a variant null (conventionally, we use lower case to refer to a variant
null). For all other cases, including the input is a SQL NULL, we return false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
