The GitHub Actions job "Flink Connector Kafka CI" on 
flink-connector-kafka.git/FLINK-40586-signal-no-more-splits-once has succeeded.
Run started by GitHub user vbhanuchander-lang (triggered by MartijnVisser).

Head commit for run:
153b64f304d5ba42be666ae29bc0cdf8ad4e4fac / Bhanu Chander Vallabaneni 
<[email protected]>
[FLINK-40586][Connectors/Kafka] Clear the no-more-splits dedup set on a 
metadata change

FLINK-39712 added readersWithNoMoreSplits so a reader is told no more splits
once per registration, and resets it in addReader. A metadata change is the
other point at which those records stop describing reality:
onHandleSubscribedStreamsFetch closes and recreates every sub enumerator, each
one signals again for its new assignments, and DynamicKafkaSourceReader has
closed and recreated its sub readers, so every sub reader is back to
noMoreSplitsAssignment == false.

Without clearing the set at that point it suppresses the second round of
signals, and a bounded job does not finish after a metadata change.

Clearing happens immediately after closeAllEnumeratorsAndContexts(), which is
where the previous generation stops existing, rather than keying the set on a
generation counter: the clear sits exactly where the generation changes and
needs no extra state.

The test drives a cluster switchover, where no sub enumerator is retained.
A retained cluster cannot show this, because its partitions are already in the
restored state, so KafkaSourceEnumerator#checkPartitionChanges finds an empty
change and returns before it would signal no more splits again -- the
FLINK-31006 gap. The test fails on main with the readers signalled once instead
of twice.

Generated-by: Claude Code (Opus 5)

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/36171942459

With regards,
GitHub Actions via GitBox

Reply via email to