Github user edlee123 commented on the issue:
https://github.com/apache/spark/pull/18378
Ok I see, I can see part of the rationale is performance (from discussion
of astype above) and consistency with pyarrow
https://arrow.apache.org/docs/python/pandas.html
I guess without knowing much about the work with Arrow I was expecting it
to be consistent with how pandas converts python types e.g in Spark 2.2
What happens with Double and DateType?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
