robinrvm opened a new pull request, #29647: URL: https://github.com/apache/flink/pull/29647
## What is the purpose of the change `BlobLibraryCacheManagerTest#testApplicationLevelJarSharing` fails from time to time with `FileAlreadyExistsException` on the application directory (https://github.com/apache/flink/actions/runs/36962185529/job/110700140845). The final check in `verifyApplicationCleanup` calls `checkFileCountForApplication`, which gets the directory via `PermanentBlobCache#getStorageLocation`. That method creates the application directory (`Files.createDirectories`). The `PermanentBlobCache` cleanup task may be deleting the same directory at that moment, and `createDirectories` then fails with `FileAlreadyExistsException`. FLINK-17463 fixed the same race for the job directory in `checkFilesExist` by building the path with `BlobUtils#getStorageLocationPath` instead. That fix didn't cover this final count check. ## Brief change log - Add `TestingBlobHelpers#checkFileCountForApplicationWithoutCreatingDirectory`, which builds the application directory path with `BlobUtils#getStorageLocationPath` (as FLINK-17463 does) and counts its files without creating it. - Use it for the final check in `BlobLibraryCacheManagerTest#verifyApplicationCleanup`. - `checkFileCountForApplication` is unchanged. ## Verifying this change This change is a test-only fix, covered by the existing `BlobLibraryCacheManagerTest` tests. None of these runs reproduced the failure, before or after the fix. Runs on my fork (robinrvm/flink): - Flink CI (beta) on this PR's commit, green: https://github.com/robinrvm/flink/actions/runs/40643099197 - Flink CI (beta) on the fix before the rebase, green: https://github.com/robinrvm/flink/actions/runs/39755689298 - Full CI on the parent commit and on the fix, all test modules and E2E groups, both green: https://github.com/robinrvm/flink/actions/runs/40525395176 - The 2 affected tests (`testApplicationLevelJarSharing`, `testDifferentApplicationsNotSharing`), 100 runs on the parent commit and 100 runs with the fix, 0 failures on both sides: https://github.com/robinrvm/flink/actions/runs/39933166571 - The same 2 tests in Flink's CI Docker image, 5 runs per side, 0 failures: https://github.com/robinrvm/flink/actions/runs/39945461254 - The same 2 tests on master in Flink's CI Docker image, 1000 runs, 0 failures: https://github.com/robinrvm/flink/actions/runs/40284007942 - The same 2 tests on the parent commit and on this PR's commit, on the current CI image, 0 failures: https://github.com/robinrvm/flink/actions/runs/40654159557 To show the race, I used a separate local program (`Repro40886.java`, not part of this PR): - A background thread stands in for the `PermanentBlobCache` cleanup task. It creates the application directory and a blob file, then deletes the directory, in a loop. - At the same time, the main thread calls the final check for 10 seconds, first with the old helper and then with the new one. Results (local, several runs): - Old line (`checkFileCountForApplication`): `FileAlreadyExistsException` on the application directory every run, after 1 to 43 attempts. - New line (`checkFileCountForApplicationWithoutCreatingDirectory`): no exception in 10 seconds, 365,604 to 499,667 attempts per run. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code 2.1.299 (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
