Re: [PR] [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression [spark]
dongjoon-hyun commented on PR #46311: URL: https://github.com/apache/spark/pull/46311#issuecomment-2089353288 Yep. Merged to master for Apache Spark 4.0.0. Thank you, @harshmotw-db . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression [spark]
dongjoon-hyun closed pull request #46311: [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression URL: https://github.com/apache/spark/pull/46311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression [spark]
harshmotw-db commented on PR #46311: URL: https://github.com/apache/spark/pull/46311#issuecomment-2089347930 Thanks for the reviews @cloud-fan @HyukjinKwon @dongjoon-hyun. The tests are completed and it is ready to be merged now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression [spark]
harshmotw-db commented on code in PR #46311:
URL: https://github.com/apache/spark/pull/46311#discussion_r1585910343
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtils.scala:
##
@@ -58,9 +59,13 @@ object VariantExpressionEvalUtils {
false
} else {
val variantValue = input.getValue
- // Variant NULL is denoted by basic_type == 0 and val_header == 0
- variantValue(0) == 0
- }
+ if(variantValue.isEmpty) {
Review Comment:
Resolved
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtils.scala:
##
@@ -58,9 +59,13 @@ object VariantExpressionEvalUtils {
false
} else {
val variantValue = input.getValue
- // Variant NULL is denoted by basic_type == 0 and val_header == 0
- variantValue(0) == 0
- }
+ if(variantValue.isEmpty) {
+throw new SparkRuntimeException("MALFORMED_VARIANT", Map.empty)
Review Comment:
Resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression [spark]
dongjoon-hyun commented on code in PR #46311:
URL: https://github.com/apache/spark/pull/46311#discussion_r1585892152
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtils.scala:
##
@@ -58,9 +59,13 @@ object VariantExpressionEvalUtils {
false
} else {
val variantValue = input.getValue
- // Variant NULL is denoted by basic_type == 0 and val_header == 0
- variantValue(0) == 0
- }
+ if(variantValue.isEmpty) {
Review Comment:
I guess this fails at `Scala` linter? Could you double-check this,
@harshmotw-db ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-45891][SQL][FOLLOW-UP] Added length check to the is_variant_null expression [spark]
dongjoon-hyun commented on code in PR #46311:
URL: https://github.com/apache/spark/pull/46311#discussion_r1585892540
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtils.scala:
##
@@ -58,9 +59,13 @@ object VariantExpressionEvalUtils {
false
} else {
val variantValue = input.getValue
- // Variant NULL is denoted by basic_type == 0 and val_header == 0
- variantValue(0) == 0
- }
+ if(variantValue.isEmpty) {
+throw new SparkRuntimeException("MALFORMED_VARIANT", Map.empty)
Review Comment:
Shall we use `QueryExecutionErrors`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
