Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21704#discussion_r201900374
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -533,15 +559,15 @@ object TypeCoercion {
case a @ CreateArray(children) if !haveSameType(children) =>
val types = children.map(_.dataType)
findWiderCommonType(types) match {
- case Some(finalDataType) => CreateArray(children.map(Cast(_,
finalDataType)))
+ case Some(finalDataType) =>
CreateArray(children.map(castIfNotSameType(_, finalDataType)))
--- End diff --
Currently the optimizer doesn't remove the cast when the difference of the
`finalDataType` is only the nullabilities of nested types.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
