wangpeibin713 commented on a change in pull request #14863:
URL: https://github.com/apache/flink/pull/14863#discussion_r579775185
##########
File path:
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/operators/deduplicate/DeduplicateFunctionHelper.java
##########
@@ -86,7 +97,9 @@ static void processLastRowOnChangelog(
RowData currentRow,
boolean generateUpdateBefore,
ValueState<RowData> state,
- Collector<RowData> out)
+ Collector<RowData> out,
+ boolean isStateTtlEnabled,
+ RecordEqualiser equaliser)
Review comment:
ok， I will add this note
##########
File path:
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/operators/deduplicate/DeduplicateFunctionHelper.java
##########
@@ -96,12 +109,20 @@ static void processLastRowOnChangelog(
currentRow.setRowKind(RowKind.INSERT);
out.collect(currentRow);
} else {
- if (generateUpdateBefore) {
- preRow.setRowKind(RowKind.UPDATE_BEFORE);
- out.collect(preRow);
+ if (!isStateTtlEnabled && equaliser.equals(preRow,
currentRow)) {
+ // currentRow is the same as preRow and state cleaning is
not enabled.
+ // We do not emit retraction and update message.
+ // If state cleaning is enabled, we have to emit messages
to prevent too early
+ // state eviction of downstream operators.
Review comment:
We may need to enable this optimization only when state ttl is disabled
to prevent too early state eviction of downstream operators.
just like the implementation in
https://github.com/apache/flink/blob/f3db4220f5c8730e065734cff16237c7743b390f/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/operators/aggregate/GroupAggFunction.java#L170
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
