Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/22610
So pyarrow just added an option when converting from Pandas to raise an
error for unsafe casts. I'd have to try it out to see if it would prevent this
case though. It's a common option when working with Pandas, so users might be
familiar with it and might be more useful to expose this as a Spark conf rather
than checking the types.
Btw, I'm working on fixing the float to boolean conversion here
https://github.com/apache/arrow/pull/2698
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
