Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19884#discussion_r157958862
--- Diff: python/pyspark/sql/udf.py ---
@@ -33,6 +33,10 @@ def _wrap_function(sc, func, returnType):
def _create_udf(f, returnType, evalType):
+ from pyspark.sql.utils import _require_minimum_pyarrow_version
+
+ _require_minimum_pyarrow_version()
--- End diff --
We can't put this here because `_create_udf` is used for `udf` (not
`pandas_udf`) as well. It is not related to pyarrow and we should be able to
define normal udf without pyarrow.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
