nzw921rx opened a new pull request, #12306:
URL: https://github.com/apache/seatunnel/pull/12306
### Purpose of this pull request
Follow up on the remaining Flink issue identified in #11503.
SchemaOperator uses one global lastProcessedEventTime and treats events
whose millisecond timestamp is less than or equal to it as outdated.
Multi-table CDC recovery can create several RestoreTableSchemaEvents in the
same millisecond, causing every event after the first to be silently skipped
and leaving those tables with stale sink-side runtime schemas.
This patch exempts RestoreTableSchemaEvent from the normal timestamp
deduplication because restore events are idempotent runtime-schema refreshes
and never execute physical DDL. The existing outdated-event behavior remains
unchanged for normal schema change events.
A regression test forces restore events for two different tables to share
the same timestamp and verifies that both are broadcast and acknowledged by the
schema coordinator.
### Does this PR introduce _any_ user-facing change?
Yes. Flink multi-table CDC jobs now restore the runtime schema for every
changed table after failover even when multiple restore events have the same
millisecond timestamp.
### How was this patch tested?
- ./mvnw spotless:apply
- ./mvnw -pl
seatunnel-translation/seatunnel-translation-flink/seatunnel-translation-flink-common
-DskipITs -Dskip.spotless=true -Dcheckstyle.skip -Dtest=SchemaOperatorTest test
- Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
- git diff --check
Maven verify was not run locally.
### Check list
* [x] No new Jar binary package is added.
* [x] Documentation is not required for this targeted correctness fix.
* [x] This change is backward compatible and does not require an
incompatible-change entry.
* [x] This is not connector code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
