JingsongLi commented on code in PR #10109:
URL: https://github.com/apache/paimon/pull/10109#discussion_r4180046397
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/FlinkConnectorOptions.java:
##########
@@ -483,6 +496,20 @@ public class FlinkConnectorOptions {
+
"${UID_PREFIX}_${TABLE_NAME}_${USER_UID_SUFFIX}. If the uid suffix is not set,
flink will "
+ "automatically generate the operator
uid, which may be incompatible when the topology changes.");
+ public static final ConfigOption<Boolean>
SINK_OPERATOR_UID_COVER_ALL_OPERATORS =
+ key("sink.operator-uid.cover-all-operators")
+ .booleanType()
+ .defaultValue(false)
+ .withDescription(
+ "If true, 'sink.operator-uid.suffix' also names
every other operator a streaming "
+ + "write adds: the row conversions, the
local merge, the compaction operators, "
Review Comment:
[P2] Include the streaming CDC operators in cover-all coverage
The new option promises to name every operator a streaming write adds, but
the public CdcSinkBuilder route still creates Side Output and Schema Evolution
without a UID. On a fixed-bucket PK table with sink.operator-uid.suffix=stable
and cover-all=true, an actual CDC StreamGraph has UID=null for both. Adding one
explicitly uid-named map upstream changes their operator hashes (Side Output
5f5dbcb6... -> f2512467..., Schema Evolution bf7a1542... -> 75147ea0...).
Building the same graph under pipeline.auto-generate-uids=false fails with no
UID assigned to operator Side Output. Thus users enabling the new option for
streaming CDC remain exposed to the same orphaned HA checkpoint entries it is
intended to prevent. Apply the new assigner to the CDC
parser/schema-evolution/conversion route and add a CDC topology case; if CDC is
intentionally excluded, explicitly scope the option/documentation instead of
guaranteeing every streaming write operator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
