cloud-fan commented on a change in pull request #25024: [SPARK-27296][SQL]
Allows Aggregator to be registered as a UDF
URL: https://github.com/apache/spark/pull/25024#discussion_r363607936
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -4231,6 +4230,63 @@ object functions {
// Scala UDF functions
//////////////////////////////////////////////////////////////////////////////////////////////
+ /**
+ * Obtains a `UserDefinedFunction` that wraps the given `Aggregator`
+ * so that it may be used with untyped Data Frames.
+ * {{{
+ * val agg = // Aggregator[IN, BUF, OUT]
+ *
+ * // declare a UDF based on agg
+ * val aggUDF = udaf(agg)
+ * val aggData = df.agg(aggUDF($"colname"))
+ *
+ * // register agg as a named function
+ * spark.udf.register("myAggName", udaf(agg))
+ * }}}
+ *
+ * @tparam IN the aggregator input type
+ * @tparam BUF the aggregating buffer type
+ * @tparam OUT the finalized output type
+ *
+ * @param agg the typed Aggregator
+ *
+ * @return a UserDefinedFunction that can be used as an aggregating
expression.
+ * The input encoder is inferred from the input type IN.
+ */
+ def udaf[IN: TypeTag, BUF, OUT](agg: Aggregator[IN, BUF, OUT]):
UserDefinedFunction = {
+ udaf(agg, ExpressionEncoder[IN]())
+ }
+
+ /**
+ * Obtains a `UserDefinedFunction` that wraps the given `Aggregator`
+ * so that it may be used with untyped Data Frames.
+ * {{{
+ * val agg = // Aggregator[IN, BUF, OUT]
Review comment:
nit: this method is mostly for java users so should be better to write the
example code in java.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
