viirya commented on issue #22807: [WIP][SPARK-25811][PySpark] Raise a proper error when unsafe cast is detected by PyArrow URL: https://github.com/apache/spark/pull/22807#issuecomment-452177528 > Were you thinking of putting in a spark config to toggle the safe flag here or in a follow up? I think we really need that because prior to v0.11.0 it allowed unsafe casts, but now raises an error by default. Do we have proper way to get spark config value at executor side like serializers here? I found that in `read_udfs` some configs for pandas_udf evaluation are read from stream, is it the only way we have?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
