Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19872#discussion_r162238002
--- Diff: python/pyspark/worker.py ---
@@ -110,6 +110,17 @@ def wrapped(*series):
return wrapped
+def wrap_pandas_group_agg_udf(f, return_type):
+ arrow_return_type = to_arrow_type(return_type)
+
+ def wrapped(*series):
+ import pandas as pd
+ result = f(*series)
+ return pd.Series(result)
--- End diff --
I think it uses `Series` to reuse the same code path about Arrow/Pandas
ser/de.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
