oscerd opened a new pull request, #29071: URL: https://github.com/apache/camel/pull/29071
Fixes [CAMEL-26229](https://issues.apache.org/jira/browse/CAMEL-26229), found while adding the catalog body types (CAMEL-24961). ## 1. The consumer marks a failed file as done **Was:** `processCollection` called `getProcessor().process(exchange)` and never checked `exchange.getException()`. A file whose route failed was marked done (with a file attribute) and the timestamp moved past it, so it was never offered again. An exception thrown by the processor was caught and ignored. **Now:** The consumer checks the exchange after the route. When it failed, the consumer reports the failure, with the exchange, to its exception handler, and leaves the file as it was, so that the next poll offers it again: - With a file attribute (`FileAttribute`, `TimeStampAndFileAttribute` and `PersistentTimestampAndFileAttribute`), the mark that the consumer set before routing the file is removed, and the poll goes on with the other files. A stored timestamp stays before the failed file, so a consumer started again still finds it. - With `TimeStamp` and `PersistentTimestamp`, the timestamp is the only record of the routed files. The files of a poll are now routed in the order of their upload date (the query is sorted by `uploadDate`), and the poll stops at a file whose exchange fails: the timestamp stays before it, and the next poll offers it again, then the files after it. When a routed file has the same upload date as the failed one, the timestamp moves to just before that date, so both are offered again. The exchange is now created without auto release and released after the consumer read its exception: with pooled exchanges, an exchange released automatically is reset before the consumer can check it. `GridFsEndpoint.createConsumer` now calls `configureConsumer`, so the `exceptionHandler` and `bridgeErrorHandler` options, which the catalog lists for the consumer, apply to the consumer that reports the failures. ## 2. The consumer never closes the download stream **Was:** The `GridFSDownloadStream` set as the body was never closed. **Now:** An on completion of the exchange closes it when the exchange is done, also when the exchange failed; a route that hands the exchange over to another thread hands the on completion over with it. The stream is now opened before the exchange is created, so a file that cannot be opened no longer leaves an exchange that is never released. ## 3. The consumer sets no CamelFileName **Was:** The consumer set the content type, metadata, length and upload date headers, but not the name of the file. **Now:** It sets `CamelFileName` (`Exchange.FILE_NAME`, as the file consumers do) to the name of the file. The metadata of `CamelFileName` now describes the consumer as well as the producer, and the description of `CamelFileLastModified` says it is the upload date (it said "The size of the file"). ## 4. An unknown operation silently does nothing **Was:** `GridFsProducer.process` was an if/else-if chain over the operation names with no final else, so `findone` succeeded and did nothing. **Now:** Another operation fails with an `IllegalArgumentException`: `Unsupported operation: findone. The supported operations are: create, remove, findOne, listAll and count`. ## 5. listAll prints the id as BsonObjectId{value=...} **Was:** Each line read `name<TAB>BsonObjectId{value=665f...}`. **Now:** Each line reads `name<TAB>665f...`, the hex string of the ObjectId, which a route can pass back in the `CamelGridFsObjectId` header. `getObjectId()` is not used, since it fails on a file that another client uploaded with a custom id: a String id is given as its text, and another custom id as its BSON value, as before. ## Tests `GridFsConsumerTimestampTest` gets 15 tests that start the consumer in a route, so that the exchange runs in a unit of work as in an application, against the module's fake bucket, which now also sorts by upload date and records the done and unset updates. They cover: a failed file offered again by each strategy with a file attribute while the poll goes on with the other files, and the failure reported to the `exceptionHandler` option; the timestamp strategies stopping at a failed file and offering it again; the files routed in the order of their upload date; the files with the same upload date as a failed file offered again; the stored timestamp of `PersistentTimestampAndFileAttribute` kept before a failed file; the `CamelFileName` header with each strategy; and the download stream closed for a completed and for a failed exchange. The new `GridFsProducerTest` (5 tests, Mockito) covers an unknown operation from the header and from the option, and the listAll ids. With the change of `src/main` reverted, all 20 new tests fail on the old behaviour (the failed file not offered again, a null `CamelFileName`, the streams not closed, no exception for `findone`, `BsonObjectId{value=...}`), and the 20 tests that were there still pass. The ITs check the `CamelFileName` header of the consumer, and the listAll id and an unknown operation against MongoDB. They need MongoDB and were not run here. Module: 40 tests, 0 failures. The consumer tests also ran three more times without retries, with no failure. ## Docs and generated files The component page gets a consumer section (the headers, the stream, and what happens when an exchange fails with each query strategy), the list of supported operations, and a listAll example with ObjectId hex strings. `mongodb-gridfs.json` and the component page are regenerated and copied to the catalog. The metadata of the `CamelFileName` and `CamelFileLastModified` headers also changes the generated endpoint DSL. The upgrade guide gets an entry. ## Not changed - With the timestamp strategies, a file that always fails holds back the files uploaded after it, since the timestamp cannot pass it without losing it. The component page says so, and points to a dead letter channel or a strategy with a file attribute, which goes on with the other files. - The sort by upload date is used only by `TimeStamp` and `PersistentTimestamp`, which need it to stop at a failed file; the strategies with a file attribute keep the order of the bucket. - Operation names stay case sensitive: `findone` fails rather than being read as `findOne`. - An error of MongoDB while marking or unmarking a file still ends the poll, which the consumer reports and retries at the next poll, as for the other poll errors. Main only, as the other fixes of the CAMEL-24961 follow-ups. _Claude Code on behalf of Andrea Cosentino (oscerd)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
