Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20295#discussion_r164347701
--- Diff: python/pyspark/sql/udf.py ---
@@ -54,7 +54,7 @@ def _create_udf(f, returnType, evalType):
"Instead, create a 1-arg pandas_udf and ignore the arg in
your function."
)
- if evalType == PythonEvalType.SQL_PANDAS_GROUP_MAP_UDF and
len(argspec.args) != 1:
+ if evalType == PythonEvalType.SQL_PANDAS_GROUP_MAP_UDF and
len(argspec.args) not in (1, 2):
raise ValueError(
"Invalid function: pandas_udfs with function type
GROUP_MAP "
"must take a single arg that is a pandas DataFrame."
--- End diff --
We should update the error message here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
