[
https://issues.apache.org/jira/browse/FLINK-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973662#comment-15973662
]
ASF GitHub Bot commented on FLINK-6091:
---------------------------------------
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/3733#discussion_r112080186
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/runtime/aggregate/GroupAggProcessFunction.scala
---
@@ -84,16 +109,38 @@ class GroupAggProcessFunction(
}
// Set aggregate result to the final output
- i = 0
- while (i < aggregates.length) {
- val index = groupings.length + i
- val accumulator = accumulators.getField(i).asInstanceOf[Accumulator]
- aggregates(i).accumulate(accumulator,
input.getField(aggFields(i)(0)))
- output.setField(index, aggregates(i).getValue(accumulator))
- i += 1
+ if (input.command == Command.Delete) {
+ i = 0
+ while (i < aggregates.length) {
+ val index = groupings.length + i
+ val accumulator =
accumulators.getField(i).asInstanceOf[Accumulator]
+ aggregates(i).retract(accumulator, input.getField(aggFields(i)(0)))
+ output.setField(index, aggregates(i).getValue(accumulator))
+ i += 1
+ }
+ } else {
+ i = 0
+ while (i < aggregates.length) {
+ val index = groupings.length + i
+ val accumulator =
accumulators.getField(i).asInstanceOf[Accumulator]
+ aggregates(i).accumulate(accumulator,
input.getField(aggFields(i)(0)))
+ output.setField(index, aggregates(i).getValue(accumulator))
+ i += 1
+ }
}
- state.update(accumulators)
+ // if previous is not null, do retraction process
+ if (null != previous) {
+ if (previous.equals(output)) {
+ // ignore same output
+ return
--- End diff --
We still need to update the state even if we do not emit new values. If we
have a max aggregation with an accumulator that holds a map of `10->1, 5->2`
and we add `8`. The new accumulator will be `10->1, 8->1, 5->2` but the
aggregation result will still be 10. If we later retract `10`, the new max
would be `5` but should be `8`.
> Implement and turn on the retraction for aggregates
> ---------------------------------------------------
>
> Key: FLINK-6091
> URL: https://issues.apache.org/jira/browse/FLINK-6091
> Project: Flink
> Issue Type: Sub-task
> Components: Table API & SQL
> Reporter: Shaoxuan Wang
> Assignee: Hequn Cheng
>
> Implement functions for generating and consuming retract messages for
> different aggregates.
> 1. add delete/add property to Row
> 2. implement functions for generating retract messages for unbounded groupBy
> 3. implement functions for handling retract messages for different aggregates.
> 4. handle retraction messages in CommonCorrelate and CommonCalc (retain
> Delete property).
> Note: Currently, only unbounded groupby generates retraction and it is
> working under unbounded and processing time mode. Hence, retraction is only
> supported for unbounded and processing time aggregations so far. We can add
> more retraction support later.
> supported now: unbounded groupby, unbounded and processing time over window
> unsupported now: group window, event time or bounded over window.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
