Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21650 @BryanCutler I think your suggestion would change the behavior. Using ArrowEvalExec and BatchEvalExec are still different when it comes to corner cases, for example, type coercion (ArrowEvalExec supports type coercion but BatchEvalExec doesn't) and timestamp type (regular UDF expects Python datetime for timestamp and pandas UDF expects pd.Timestamp) I think this is probably a good future improvement but not great for this Jira because of the behavior change. WDYT?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org