viirya commented on a change in pull request #24576: [SPARK-27671][SQL] Fix
error when casting from a nested null in a struct
URL: https://github.com/apache/spark/pull/24576#discussion_r283634266
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
##########
@@ -620,6 +620,12 @@ case class Cast(child: Expression, dataType: DataType,
timeZoneId: Option[String
// We can return what the children return. Same thing should happen in the
codegen path.
if (DataType.equalsStructurally(from, to)) {
identity
+ } else if (from == NullType) {
+ // According to `canCast`, NullType can be casted to any type.
+ // For primitive types, we don't reach here because the guard of
`nullSafeEval`.
+ // But for nested types like struct, we might reach here for nested null
type field.
+ // We won't call the returned function actually, but returns a
placeholder.
+ _ => throw new SparkException(s"should not directly cast from NullType
to $to.")
Review comment:
If reaching here, means we call the cast function for a null, unnecessarily.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
