yaooqinn commented on a change in pull request #26651: [SPARK-30008][SQL] The
dataType of collect_list/collect_set aggs should be ArrayType(_, false)
URL: https://github.com/apache/spark/pull/26651#discussion_r349968170
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala
##########
@@ -1214,6 +1215,13 @@ class CastSuite extends CastSuiteBase {
checkEvaluation(Cast(Literal(134.12), DecimalType(3, 2)), null)
}
}
+
+ test("SPARK-30008: collect_list/collect_set can cast to ArrayType not
containsNull") {
+ val list = CollectList(Literal(1))
+ assert(Cast.canCast(list.dataType, ArrayType(IntegerType, false)))
+ val set = CollectSet(Literal(1))
+ assert(Cast.canCast(set.dataType, ArrayType(StringType, false)))
+ }
Review comment:
Thanks, should be `Fix bug. i.e. casting collect_list(a) to ArrayType(_,
false) will fail **before this fix**.`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
