Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19389#discussion_r152262569
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala
---
@@ -468,18 +477,26 @@ object PartitioningUtils {
}
/**
- * Given a collection of [[Literal]]s, resolves possible type conflicts
by up-casting "lower"
- * types.
+ * Given a collection of [[Literal]]s, resolves possible type conflicts
by
+ * [[findWiderTypeForPartitionColumn]].
*/
private def resolveTypeConflicts(literals: Seq[Literal], timeZone:
TimeZone): Seq[Literal] = {
- val desiredType = {
- val topType =
literals.map(_.dataType).maxBy(upCastingOrder.indexOf(_))
- // Falls back to string if all values of this column are null or
empty string
- if (topType == NullType) StringType else topType
- }
+ val litTypes = literals.map(_.dataType)
+ val desiredType =
litTypes.fold[DataType](NullType)(findWiderTypeForPartitionColumn)
--- End diff --
nit: I think we can use `reduce`? `literals` should not be empty.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
