Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r144168563
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -31,7 +31,8 @@ object ArrowUtils {
// todo: support more types.
- def toArrowType(dt: DataType): ArrowType = dt match {
+ /** Maps data type from Spark to Arrow. NOTE: timeZoneId required for
TimestampTypes */
+ def toArrowType(dt: DataType, timeZoneId: Option[String]): ArrowType =
dt match {
--- End diff --
I don't think it's a good idea to have a default value for `timeZoneId`,
then it makes it very easy to forget it and miss places in the code that really
should have a timezone. It will compile and probably pass tests, but might
fail somewhere down the line. Any place in Spark should specify a timezone, so
it's better to force an argument to be used. Only in the tests, where we are
sure no TimestampTypes are used would we set it to None.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
