rkhachatryan commented on code in PR #27602:
URL: https://github.com/apache/flink/pull/27602#discussion_r2841510205
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/stream/StreamExecSink.java:
##########
@@ -383,13 +400,30 @@ protected Transformation<RowData> applyUpsertMaterialize(
private OneInputStreamOperator<RowData, RowData> createSumOperator(
ExecNodeConfig config,
RowType physicalRowType,
+ int[] primaryKeys,
+ String[] pkFieldNames,
int[] inputUpsertKey,
GeneratedRecordEqualiser upsertKeyEqualiser,
GeneratedHashFunction upsertKeyHashFunction,
StateTtlConfig ttlConfig,
GeneratedRecordEqualiser rowEqualiser,
GeneratedHashFunction rowHashFunction) {
+ // Check if we should use the watermark-compacting materializer for
ERROR/NOTHING strategies
+ if (isErrorOrNothingConflictStrategy()) {
+ RowType keyType = RowTypeUtils.projectRowType(physicalRowType,
primaryKeys);
+
+ return WatermarkCompactingSinkMaterializer.create(
+ conflictStrategy,
+ physicalRowType,
+ rowEqualiser,
+ upsertKeyEqualiser,
+ inputUpsertKey,
+ keyType,
+ pkFieldNames);
+ }
+
+ // Use existing logic for DEDUPLICATE (legacy behavior)
Review Comment:
That would allow the default behavior to benefit from compaction and reduce
the number of code paths.
However, it would also require state migration which is probably worse in
terms of risks and complexity 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
