Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19884#discussion_r157960643
--- Diff: python/pyspark/sql/udf.py ---
@@ -33,6 +33,10 @@ def _wrap_function(sc, func, returnType):
def _create_udf(f, returnType, evalType):
+ from pyspark.sql.utils import _require_minimum_pyarrow_version
+
+ _require_minimum_pyarrow_version()
--- End diff --
Yeah, that is not good! I was a little hesitant to put it in `def
pandas_udf` because things are a little different when used as a decorator.
How about leave it in `_create_udf` only when the eval type is a Pandas form?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
