Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/3423#discussion_r103475723
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/runtime/aggregate/DataSetTumbleTimeWindowAggReduceCombineFunction.scala
---
@@ -28,68 +30,74 @@ import org.apache.flink.types.Row
* [[org.apache.flink.api.java.operators.GroupCombineOperator]].
* It is used for tumbling time-window on batch.
*
- * @param rowtimePos The rowtime field index in input row
- * @param windowSize Tumbling time window size
- * @param windowStartPos The relative window-start field position to the
last field of output row
- * @param windowEndPos The relative window-end field position to the last
field of output row
- * @param aggregates The aggregate functions.
+ * @param windowSize Tumbling time window size
+ * @param windowStartPos The relative window-start field position to
the last field of output row
+ * @param windowEndPos The relative window-end field position to the
last field of output row
+ * @param aggregates The aggregate functions.
* @param groupKeysMapping The index mapping of group keys between
intermediate aggregate Row
* and output Row.
* @param aggregateMapping The index mapping between aggregate function
list and aggregated value
* index in output Row.
- * @param intermediateRowArity The intermediate row field count
- * @param finalRowArity The output row field count
+ * @param finalRowArity The output row field count
*/
class DataSetTumbleTimeWindowAggReduceCombineFunction(
- rowtimePos: Int,
windowSize: Long,
windowStartPos: Option[Int],
windowEndPos: Option[Int],
- aggregates: Array[Aggregate[_ <: Any]],
+ aggregates: Array[AggregateFunction[_ <: Any]],
groupKeysMapping: Array[(Int, Int)],
aggregateMapping: Array[(Int, Int)],
- intermediateRowArity: Int,
finalRowArity: Int)
extends DataSetTumbleTimeWindowAggReduceGroupFunction(
- rowtimePos,
windowSize,
windowStartPos,
windowEndPos,
aggregates,
groupKeysMapping,
aggregateMapping,
- intermediateRowArity,
finalRowArity)
- with CombineFunction[Row, Row] {
+ with CombineFunction[Row, Row] {
/**
* For sub-grouped intermediate aggregate Rows, merge all of them into
aggregate buffer,
*
- * @param records Sub-grouped intermediate aggregate Rows iterator.
+ * @param records Sub-grouped intermediate aggregate Rows iterator.
* @return Combined intermediate aggregate Row.
*
*/
override def combine(records: Iterable[Row]): Row = {
- // initiate intermediate aggregate value.
- aggregates.foreach(_.initiate(aggregateBuffer))
-
- // merge intermediate aggregate value to buffer.
var last: Row = null
-
val iterator = records.iterator()
+ val accumulatorList = Array.fill(aggregates.length) {
+ new JArrayList[Accumulator]()
+ }
+
+ // per each aggregator, collect its accumulators to a list
while (iterator.hasNext) {
val record = iterator.next()
- aggregates.foreach(_.merge(record, aggregateBuffer))
+ for (i <- aggregates.indices) {
+ accumulatorList(i).add(
--- End diff --
pairwise merging
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
