Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/18664
I agree with Bryan. I think we might want to rethink the assumption that
toPandas result with arrow / without arrow should be 100% the same.
For instance, non-Arrow doesn't respect session local timezone, and if for
compatible reasons that we cannot fix this in the non-Arrow version, then
let's fix it in the Arrow version and document the difference. IMHO,
keeping a new feature bug compatible of the existing feature is not
necessarily, fixing non ideal behavior in the new feature it provides a
migration path off the buggy behavior of existing feature.
On Thu, Oct 5, 2017 at 7:34 PM Bryan Cutler <[email protected]>
wrote:
> @ueshin <https://github.com/ueshin> @HyukjinKwon
> <https://github.com/hyukjinkwon> , I think it would be critical for users
> to have timestamps working for Arrow. Just to recap, the remaining issue
> here was that toPandas() without Arrow does not have timestamps with a
> timezone. Is it possible we can document that difference and not hold this
> up for fixing the case without Arrow? Arrow is still disabled by default,
> so the default behavior of toPandas() does not change.
>
> â
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/18664#issuecomment-334620292>, or
mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AAwbrGUKC2aM-9gfFvLlw_SN2SUwrFQQks5spWehgaJpZM4Oateu>
> .
>
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
