yuxiqian opened a new pull request, #3912:
URL: https://github.com/apache/flink-cdc/pull/3912
This closes FLINK-37278.
Currently, regular SE topology uses the following POC to drain existing
`DataChangeEvent`s in the pipeline:
1. SchemaOperator ("client") emits `FlushEvent` to downstream.
2. The "client" keeps polling the SchemaCoordinator ("server") with 1-second
interval.
3. The "server" rejects all requests from clients until it has collected
enough `FlushSuccessEvent` notifications from Sink.
As a result, all schema change requests will took at least 1 second to
finish, after at least one polling interval.
---
This PR replaces the polling code with maintaining a pending schema change
request queue, where SchemaCoordinator could manage all pending clients and
effectively blocking them from handling upstream events. Schema evolution
process could start immediately after `FlushSuccessEvent` got reported,
needless to wait for polling requests from clients.
---
With this change, time consumption of
`testRegularTablesSourceInMultipleParallelism` test case has been reduced from
~6 minutes to ~50 seconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
