The GitHub Actions job "Flink Connector Kafka CI" on 
flink-connector-kafka.git/FLINK-40586-signal-no-more-splits-once has failed.
Run started by GitHub user vbhanuchander-lang (triggered by MartijnVisser).

Head commit for run:
557001f3dc6e407d47625437178e2decd9aa669f / Bhanu Chander Vallabaneni 
<[email protected]>
[FLINK-40586][Connectors/Kafka] Signal no more splits to each reader once per 
registration

DynamicKafkaSourceEnumerator.handleNoMoreSplits() signalled every registered
reader on every invocation, and it is reached from addReader, addSplitsBack and
tryCompletePendingReaderRegistration. A reader that already finished on an
earlier signal is therefore signalled again when another reader registers.
NoMoreSplitsEvent is not loss tolerant, so delivering it to a FINISHED task
fails the job with "An OperatorEvent from an OperatorCoordinator to a task was
lost", which is the DynamicKafkaSourceITTest.testIdleReader failure seen in CI.

Track which subtasks have been signalled and skip them, clearing a subtask's
entry in addReader so a reader that re-registers after failover is signalled
again. Keying on registration rather than on addSplitsBack matters for an idle
reader: it has no splits to hand back, so it would otherwise never be signalled
again after a restart and the job would not finish.

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/34427072795

With regards,
GitHub Actions via GitBox

Reply via email to