Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20531#discussion_r166656777
--- Diff: python/pyspark/sql/tests.py ---
@@ -4509,23 +4523,32 @@ def weighted_mean(v, w):
return weighted_mean
def test_manual(self):
+ from pyspark.sql.functions import pandas_udf, array
+
df = self.data
sum_udf = self.pandas_agg_sum_udf
mean_udf = self.pandas_agg_mean_udf
-
- result1 = df.groupby('id').agg(sum_udf(df.v),
mean_udf(df.v)).sort('id')
+ mean_arr_udf = pandas_udf(
+ self.pandas_agg_mean_udf.func,
--- End diff --
If you meant to type coercion (did I understand correctly?), I already
tested. Seems not working properly. Similar thing was discussed in
https://github.com/apache/spark/pull/20163#issuecomment-356231655 (thanks
@ueshin).
Will reread the comments when I am more awake tomorrow ...
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
