Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20567
The binary type bug sounds like a blocker, can we just fix it surgically by
checking the supported data types before going to the arrow optimization path?
For now we can stick with that the current behavior is, i.e. throw exception.
The inconsistent behavior between `toPandas` and `createDataFrame` is
confusing but may not be a blocker. We can fix it in Spark 2.4 and add a note
in the migration guide.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
