The GitHub Actions job "Flink Connector Kafka CI" on 
flink-connector-kafka.git/v3.4 has succeeded.
Run started by GitHub user MartijnVisser (triggered by MartijnVisser).

Head commit for run:
22310e625ecfe352a217035297f90d1d8ec10e63 / Sylwester Lachiewicz 
<[email protected]>
[FLINK-40362][connector/kafka] Fix idle reader hang when no-more-splits 
precedes metadata update (#291)

An idle reader (subtask with no assigned splits) that receives the 
no-more-splits
signal before the reply to GetMetadataUpdateEvent never forwards it to the 
sub-readers
created by that update: the re-notification in handleSourceEvents was guarded by
!pendingSplits.isEmpty(), which is false in exactly this case. The reader then
returns NOTHING_AVAILABLE indefinitely and a bounded job never finishes.

Re-deliver the no-more-splits signal after processing the reader's first 
metadata update
(captured via !isActivelyConsumingSplits before the flag flips). Replaying only 
on the
first metadata update ensures that an active reader waiting for splits on newly 
added
clusters/topics does not terminate prematurely before those splits and the 
subsequent
no-more-splits signal arrive. notifyNoMoreSplits() is idempotent and only fires 
when
isNoMoreSplits is true.

Verified:
- 
DynamicKafkaSourceReaderTest#testIdleReaderFinishesWhenNoMoreSplitsArrivesBeforeMetadata:
  fails on main (NOTHING_AVAILABLE), passes with this change (END_OF_INPUT).
- 
DynamicKafkaSourceReaderTest#testActiveReaderWaitsForNewSplitsAfterMetadataChange:
  passes on main, fails if re-delivery is unconditional on every metadata 
update,
  passes with the firstMetadataUpdate guard.
- SourceTestSuiteBase#testIdleReader through 
DynamicKafkaSourceITTest$IntegrationTests
  hangs on unfixed main whenever a run leaves a subtask idle.

Generated-by: Claude Fable 5
(cherry picked from commit 48ddbf1fd5d6d3d60ed30ec4ab0780f70f9cbc51)

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/34612527605

With regards,
GitHub Actions via GitBox

Reply via email to