The GitHub Actions job "Flink Connector Kafka CI" on 
flink-connector-kafka.git/fix-enumerator-duplicate-partition-assignment has 
failed.
Run started by GitHub user slachiewicz (triggered by slachiewicz).

Head commit for run:
272e7ef7a20f586c2d3d184e8f7a33788e9a913f / Sylwester Lachiewicz 
<[email protected]>
[FLINK-40615][Connectors/Kafka] Fix duplicate partition assignment in 
KafkaSourceEnumerator when discovery interval is shorter than split 
initialization

KafkaSourceEnumerator.getPartitionChange() only deduplicates against
assignedSplits and pendingPartitionSplitAssignment. When periodic
discovery fires faster than initializePartitionSplits completes on the
worker thread, the same partitions are treated as new again, causing
duplicate split assignments and duplicate record consumption.

Fix: track in-flight partitions in a partitionsBeingInitialized set,
populated before dispatching callAsync, cleared on handler completion
(both success and failure), and excluded from getPartitionChange dedup.

SourceTopicIntegrityTest keeps its 50ms discovery interval; that
interval is what surfaced the race, so it stays as the regression
guard rather than being tuned away.

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/34462949658

With regards,
GitHub Actions via GitBox

Reply via email to