MartijnVisser opened a new pull request, #29386:
URL: https://github.com/apache/flink/pull/29386
## What is the purpose of the change
`SingleThreadFetcherManager#getRunningFetcher()` checks `fetchers.isEmpty()`
and then reads the first value. A fetcher thread can remove itself from the map
in between, so `addSplits()` fails with `NoSuchElementException`.
This is Kim Gräsman's fix from #25340. Becket approved the fix and asked for
a test; the stale bot closed the PR after Kim added one. I rebased it and
reworked the test.
## Brief change log
- Take the running fetcher from a single iterator.
- Add a repeated test for `addSplits()` while the manager closes.
## Verifying this change
`SplitFetcherManagerTest#testAddSplitsWhileClosing` failed in all 100 class
runs without the fix (2998 of 10000 repetitions) and in none of 1000 with it,
under CPU load. It closes the manager because that reliably makes fetcher
threads remove themselves; how the reader in the Jira hit it while running is
unclear.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
