Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18659#discussion_r137402913
--- Diff: python/pyspark/sql/functions.py ---
@@ -2112,7 +2113,7 @@ def wrapper(*args):
@since(1.3)
-def udf(f=None, returnType=StringType()):
+def udf(f=None, returnType=StringType(), vectorized=False):
--- End diff --
@felixcheung does this fit your idea for a more generic decorator? Not
exclusively labeled as `pandas_udf`, just enable vectorization with a flag,
e.g. `@udf(DoubleType(), vectorized=True)`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
