allthingssecurity opened a new pull request, #29290: URL: https://github.com/apache/camel/pull/29290
# Description [CAMEL-25960](https://issues.apache.org/jira/browse/CAMEL-25960) Follow-up of the review of #28255 (CAMEL-25772). With a done file with a fixed name (`doneFileName=ready`), a poll that leaves files of a directory keeps the done file, so its last file does not delete it. A poll that lists the directory completely cleared the kept set in `GenericFileKeptDoneFiles.startBatch`, and kept the done file of the files it leaves again only later: in `poll()` for the non-eager `maxMessagesPerPoll` cut (after sorting), and in `processBatch` for a file that is not started or is left when the batch stops. When the last file of the previous poll completed on another thread in between (the route continues with `threads()`, `seda` or an asynchronous producer), `GenericFileOnCompletion.handleDoneFile` found the done file not kept and deleted it, and the files the new poll left waited for a new done file. Change: - `GenericFileKeptDoneFiles.Batch`: the kept done files as the files of one poll see them. `processExchange` hands each file the batch of its poll. The last file of a poll keeps the done file when its own poll kept it, or when the current poll keeps it (as before, so a done file kept by a newer poll is not deleted by an older file). - A complete listing starts a new set for the new poll instead of clearing the one the files of the previous poll still check. The done files returned as "gone" are computed as before, and a cut listing still leaves the set as it is (CAMEL-25772). Of the two fixes the JIRA suggests, this is the second (keep the old entries until the poll has decided): the files a poll leaves are known only while its batch runs (a file that is not started), so the kept set of a poll cannot be computed before it is replaced. The reproduction outline in the JIRA needs 5 files, not 3, so that the second poll leaves a file too. No behaviour change for synchronous routes: there the last file of a poll completes before the next poll starts. No option, no upgrade-guide entry. Tests: - `FileConsumerDoneFileNameAsyncLastFileTest` (camel-core): `maxMessagesPerPoll=2`, `eagerMaxMessagesPerPoll=false`, 5 files, the route continues with `threads()`. The `sorter` runs on the poll thread between the complete listing and the re-keep: in the second poll it lets b.txt, the last file of the first poll, complete there (latches and a `NotifyBuilder`, no sleeps), then checks the done file. On main the done file is deleted and e.txt is never consumed; with the change it stays until the third poll consumed e.txt, which then deletes it. - `GenericFileKeptDoneFilesTest`: a unit test of the batch view across a complete listing. - The new test fails on main code and passes with the change; 20x with `FileConsumerDoneFileNameFilesLeftTest`: 300/300; `GenericFileKeptDoneFilesTest` 20x: 60/60. camel-core `org.apache.camel.component.file` tests and camel-file tests (offline): 482 run, 0 failures. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-25960](https://issues.apache.org/jira/browse/CAMEL-25960). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built camel-file, camel-core and the modules they depend on, including the formatter and import-sort plugins, and ran their file tests. No generated file changes. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
