ulysses-you commented on code in PR #40915:
URL: https://github.com/apache/spark/pull/40915#discussion_r1175192930
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectAggregationIterator.scala:
##########
@@ -111,25 +111,17 @@ class ObjectAggregationIterator(
}
}
+ private lazy val bufferFieldTypes =
+ aggregateFunctions.flatMap(_.aggBufferAttributes.map(_.dataType))
+
// Creates a new aggregation buffer and initializes buffer values. This
function should only be
- // called under two cases:
- //
- // - when creating aggregation buffer for a new group in the hash map, and
- // - when creating the re-used buffer for sort-based aggregation
+ // called when creating aggregation buffer for a new group in the hash map.
private def createNewAggregationBuffer(): SpecificInternalRow = {
- val bufferFieldTypes =
aggregateFunctions.flatMap(_.aggBufferAttributes.map(_.dataType))
val buffer = new SpecificInternalRow(bufferFieldTypes)
- initAggregationBuffer(buffer)
Review Comment:
it's the duplicate code and the code has poor performance due to the scala
sugar. `AggregationIterator.initializeBuffer` is effective and also used by
`SortBasedAggregationIterator`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
