Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
> I'm just wondering what if we use timestamp in nested types. Currently we
don't support nested types but in the future?
I'll try to take this into account, or at least add a note for timestamps
in nested types
> BTW, could we just use DateTimeUtils.defaultTimeZone() instead of
SQLConf.SESSION_LOCAL_TIMEZONE if you really meant it? I think the previous
behaviour without Arrow does not count SQLConf.SESSION_LOCAL_TIMEZONE?
@ueshin do you agree to just use `DateTimeUtils.defaultTimeZone()`? Most
places where Arrow is used this would be the case. The only scenario I can
think of that would not use it is if the user generated timestamps with
SparkSQL and then called `toPandas()` or ran a `pandas_udf`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
