zhangshenghang opened a new pull request, #12162:
URL: https://github.com/apache/seatunnel/pull/12162
## Purpose
The current `hasCheckpointedCompletionState` predicate in
`SnapshotSplitAssigner` correctly skips re-enqueueing a split whose
checkpointed completion state is fully durable, but it still re-enqueues a
split that is finished in the reader state but missing the corresponding
watermark in the enumerator state.
That state gap is real: a reader that marks a split `snapshot-read-finished`
and then crashes (or is preempted by a static-slot failover) before its next
`CompletedSnapshotSplitsReportEvent` goes out leaves the enumerator without a
watermark. The assigner keeps waiting for the report that the dead reader will
never send, and the snapshot phase can never complete.
## Changes
- `SnapshotSplitAssigner`:
- Rename `hasCheckpointedCompletionState` to
`restoreCompletedSnapshotSplit` and have it rebuild the missing
`SnapshotSplitWatermark` from the split's own low/high watermark via
`putIfAbsent`. The split is then skipped on add-back, and the assigner can move
on.
- Unfinished or unknown splits are still re-enqueued for replay.
- Refresh the log message to reflect the new behavior.
- `SnapshotSplitAssignerTest`: update
`testAddSplitsShouldReplayFinishedSplitWithoutCompletedWatermark` to
`testAddSplitsShouldRestoreFinishedSplitWithoutCompletedWatermark`. The new
assertions expect the finished-but-unwatermarked split to be restored, not
replayed, and that the reconstructed watermark is part of the next snapshot.
This is a follow-up to upstream #10859 (`[Fix][CDC] Avoid replaying
completed snapshot splits after restore`). It tightens the same code path for
the case where the checkpointed completion state is only partially durable.
## Validation
```
./mvnw -pl seatunnel-connectors-v2/connector-cdc/connector-cdc-base \
-Dtest=SnapshotSplitAssignerTest \
-Dcheckstyle.skip -Dspotless.check.skip test
```
Result: `Tests run: 3, Failures: 0, Errors: 0, Skipped: 0`
## Impact
- Behavior change: a snapshot split that is finished in the reader state but
missing the corresponding `SnapshotSplitWatermark` in the enumerator state is
now treated as completed instead of being re-enqueued for replay. This is the
right thing for static-slot failover and reader-crash-mid-snapshot scenarios; a
fully-durable completion state was already handled by upstream #10859.
- No new public API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
