maropu commented on a change in pull request #28087: [SPARK-31319][SQL][DOCS]
Document UDFs/UDAFs in SQL Reference
URL: https://github.com/apache/spark/pull/28087#discussion_r406648910
##########
File path: docs/sql-ref-functions-udf-aggregate.md
##########
@@ -1,22 +1,133 @@
---
layout: global
-title: User defined Aggregate Functions (UDAF)
-displayTitle: User defined Aggregate Functions (UDAF)
+title: User Defined Aggregate Functions (UDAFs)
+displayTitle: User Defined Aggregate Functions (UDAFs)
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---
-**This page is under construction**
+### Description
+
+User-Defined Aggregate Functions (UDAFs) are user-programmable routines that
act on multiple rows at once and return a single aggregated value as a result.
This documentation lists the classes that are required for creating and
registering UDAFs. It also contains examples that demonstrate how to define and
register UDAFs in Scala and invoke them in Spark SQL.
+
+### Aggregator[-IN, BUF, OUT]
+
+A base class for user-defined aggregations, which can be used in Dataset
operations to take all of the elements of a group and reduce them to a single
value.
+- IN The input type for the aggregation.
+
+- BUF The type of the intermediate value of the reduction.
+
+- OUT The type of the final output result.
+
+<dl>
+ <dt><code><em>bufferEncoder: Encoder[BUF]</em></code></dt>
+ <dd>
+ Specifies the Encoder for the intermediate value type.
+ </dd>
+</dl>
+
+
+<dl>
+ <dt><code><em>finish(reduction: BUF): OUT</em></code></dt>
+ <dd>
+ Transform the output of the reduction.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>merge(b1: BUF, b2: BUF): BUF</em></code></dt>
+ <dd>
+ Merge two intermediate values.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>outputEncoder: Encoder[OUT]</em></code></dt>
+ <dd>
+ Specifies the Encoder for the final output value type.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>reduce(b: BUF, a: IN): BUF</em></code></dt>
+ <dd>
+ Aggregate input value <code>a</code> into current intermediate value. For
performance, the function may modify <code>b</code> and return it instead of
constructing new object for <code>b</code>.
+ </dd>
+</dl>
+
+<dl>
+ <dt><code><em>zero: BUF</em></code></dt>
+ <dd>
+ The initial value of the intermediate result for this aggregation.
+ </dd>
+</dl>
+
+### UDFRegistration
+
+Functions for registering user-defined functions. Use `SparkSession.udf` to
access this.
Review comment:
I still think its ok for users to just know how to register udf/udaf in the
document, so user don't need to know what `UDFRegistration` is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
