Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138831092
--- Diff: python/pyspark/sql/functions.py ---
@@ -2111,6 +2126,53 @@ def wrapper(*args):
return wrapper
+def _udf(f, returnType, vectorized):
+ udf_obj = UserDefinedFunction(f, returnType, vectorized=vectorized)
+ return udf_obj._wrapped()
+
+
+if _have_pandas and _have_arrow:
+
+ @since(2.3)
+ def pandas_udf(f=None, returnType=StringType()):
--- End diff --
Instead of hiding `pandas_udf` when no pandas and arrow installed, should
we throw a exception if users without pandas and arrow try to use it?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
