I don't believe UDAFs are available in PySpark as this came up on the developer list while I was asking for what features people were missing in PySpark - see http://apache-spark-developers-list.1001551.n3.nabble.com/Python-Spark-Improvements-forked-from-Spark-Improvement-Proposals-td19422.html . The JIRA for tacking this issue is at https://issues.apache.org/jira/browse/SPARK-10915
On Sat, Oct 15, 2016 at 7:20 PM, Tobi Bosede <ani.to...@gmail.com> wrote: > Hello, > > I am trying to use a UDF that calculates inter-quartile (IQR) range for > pivot() and SQL in pyspark and got the error that my function wasn't an > aggregate function in both scenarios. Does anyone know if UDAF > functionality is available in python? If not, what can I do as a work > around? > > Thanks, > Tobi > -- Cell : 425-233-8271 Twitter: https://twitter.com/holdenkarau
