Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21982#discussion_r208491880
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
---
@@ -270,3 +278,90 @@ case class ArrayFilter(
override def prettyName: String = "filter"
}
+
+/**
+ * Applies a binary operator to a start value and all elements in the
array.
+ */
+@ExpressionDescription(
+ usage =
+ """
+ _FUNC_(expr, start, merge, finish) - Applies a binary operator to an
initial state and all
+ elements in the array, and reduces this to a single state. The final
state is converted
+ into the final result by applying a finish function.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(array(1, 2, 3), (acc, x) -> acc + x);
--- End diff --
@ueshin, would you mind if I ask to kindly double check if the example
works? seems not in my local:
```
spark-sql> SELECT aggregate(array(1, 2, 3), (acc, x) -> acc + x);
2018-08-08 16:08:25 ERROR SparkSQLDriver:91 - Failed in [SELECT
aggregate(array(1, 2, 3), (acc, x) -> acc + x)]
org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to
dataType on unresolved object, tree: 'acc
at
org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute.dataType(unresolved.scala:105)
at
org.apache.spark.sql.catalyst.expressions.BinaryArithmetic.dataType(arithmetic.scala:119)
at
org.apache.spark.sql.catalyst.expressions.LambdaFunction.dataType(higherOrderFunctions.scala:72)
at
org.apache.spark.sql.hive.HiveSessionCatalog$$anonfun$1.apply(HiveSessionCatalog.scala:122)
at
org.apache.spark.sql.hive.HiveSessionCatalog$$anonfun$1.apply(HiveSessionCatalog.scala:121)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
