lvyanquan commented on code in PR #4418:
URL: https://github.com/apache/flink-cdc/pull/4418#discussion_r3987429064
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/assigners/MySqlHybridSplitAssigner.java:
##########
@@ -161,20 +194,76 @@ public void addSplits(Collection<MySqlSplit> splits) {
} else {
// we don't store the split, but will re-create binlog split
later
isBinlogSplitAssigned = false;
+ // re-creating the binlog split: the reader must re-assemble
and re-report
+ // before the snapshot metadata can be released again. Bumping
the generation
+ // invalidates any assembled event still in flight from the
failed attempt.
+ binlogSplitMetaAssembled = false;
+ checkpointIdToReleaseMeta = null;
+ binlogAssignmentGeneration++;
Review Comment:
This only handles cases where the binlog split is actually added back. If
its assignment was already checkpointed, a reader reset may call
addSplitsBack() with an empty list, so the generation is not incremented even
though the reader
may restore an incomplete metadata state. A delayed assembled event from the
failed attempt could then trigger an unsafe release.
Could we also reset the assembled state and increment the generation
whenever the binlog reader is reset, including empty add-back cases, and add a
test for this scenario?
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceConfigFactory.java:
##########
Review Comment:
Could we fail fast when both options are enabled instead of silently
disabling metadata release here?
```
Preconditions.checkState(
!(scanNewlyAddedTableEnabled && releaseSnapshotMetadataEnabled),
"scan.incremental.snapshot.metadata.release.enabled and "
+ "scan.newly-added-table.enabled cannot both be enabled.");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
