Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22913#discussion_r230698731
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -71,6 +71,7 @@ object ArrowUtils {
case d: ArrowType.Decimal => DecimalType(d.getPrecision, d.getScale)
case date: ArrowType.Date if date.getUnit == DateUnit.DAY => DateType
case ts: ArrowType.Timestamp if ts.getUnit == TimeUnit.MICROSECOND =>
TimestampType
+ case date: ArrowType.Date if date.getUnit == DateUnit.MILLISECOND =>
TimestampType
--- End diff --
Yea, but date is a date. Otherwise, users should use timestamp types if
they don't want to truncate. I don't think we should switch the type to cover
its capability.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
