cloud-fan commented on a change in pull request #24144: [SPARK-24935][SQL] fix
Hive UDAF with two aggregation buffers
URL: https://github.com/apache/spark/pull/24144#discussion_r266854826
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
##########
@@ -356,33 +356,23 @@ private[hive] case class HiveUDAFFunction(
@transient
private lazy val partial2ModeEvaluator = {
val evaluator = newEvaluator()
- evaluator.init(GenericUDAFEvaluator.Mode.PARTIAL2,
Array(partial1HiveEvaluator.objectInspector))
- evaluator
+ HiveEvaluator(evaluator, evaluator.init(
+ GenericUDAFEvaluator.Mode.PARTIAL2,
Array(partial1HiveEvaluator.objectInspector)))
}
// Spark SQL data type of partial aggregation results
@transient
private lazy val partialResultDataType =
inspectorToDataType(partial1HiveEvaluator.objectInspector)
- // The UDAF evaluator used to compute the final result from a partial
aggregation result objects.
- // Hive `ObjectInspector` used to inspect the final aggregation result
object.
- @transient
- private lazy val finalHiveEvaluator = {
Review comment:
This is only used to serialize the final result, which can be done by
partial2 evaluator as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
