[
https://issues.apache.org/jira/browse/CAMEL-25681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25681 started by shashank.
----------------------------------------
> camel-support - FileIdempotentRepository.add keeps a key in its cache when
> the file store cannot be read or written, so the retry of that message is
> dropped as a duplicate
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25681
> URL: https://issues.apache.org/jira/browse/CAMEL-25681
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> {{FileIdempotentRepository.add(key)}} (main cd2ce899764e, lines 145-176) puts
> the key into its 1st-level cache first, then checks the file store
> ({{containsStore}}) and appends the key to it ({{appendToStore}}). When the
> store cannot be read or written (disk full, permissions, an I/O error of a
> network file system) these methods throw a {{RuntimeCamelException}}, {{add}}
> throws, and the message fails - but the key stays in the cache. Every later
> {{add}} of the same key returns {{false}} from the cache, until the
> application is restarted (a key that was never stored is then gone from the
> cache too).
> Consequence: the redelivery of the failed message is taken for a duplicate
> and skipped, also after the disk has space again. For example, an
> {{idempotentConsumer}} with a {{FileIdempotentRepository}} after a JMS
> consumer: during a "disk full" period every message fails (logged), the
> broker redelivers them, and the redeliveries are filtered as duplicates and
> acknowledged - lost without a further trace. (The file consumer with
> {{idempotentRepository}} calls {{add}} while it polls; what it does with the
> file after such a failure was not checked.)
> {{KafkaIdempotentRepository.add}} (camel-kafka, lines 352-363 on main)
> already removes the key from its cache again when storing it (broadcasting it
> on the topic) fails. The other repositories of the project have one store
> only.
> h3. Reproduction
> New {{FileIdempotentRepositoryFailedStoreTest}} (camel-support):
> * {{appendToStore}} fails once ("No space left on device"), then works:
> {{[the retry of a key that was not stored] Expecting value to be true but was
> false}};
> * the store cannot be read (a directory took its place), then it can: the
> same failure.
> * A third test (passes on main too) pins that a key found in the store is
> still put into the cache.
> h3. Proposed fix
> Put the key into the cache only after it was found in the store or appended
> to it. The order of the store accesses does not change, and a key found in
> the store is still cached (CAMEL-12148). No option or behaviour change for a
> store that works, so no upgrade guide entry.
> Tests with the fix: camel-support 136 run, 0 failures; camel-core tests
> matching {{*Idempotent*}}: 77 run, 0 failures. Not run: the full camel-core
> suite.
> Not changed: when {{trunkStore}} (the maximum file size is reached) fails
> after the key was appended, {{add}} still throws although the key is stored,
> so the retry is a duplicate; and {{remove}} removes the key from the cache
> before the store, so a failing store keeps it in the file (but {{contains}}
> reads the file and still answers {{true}}).
> How it was found: a review of the repositories that keep a cache in front of
> a store (backlog item of an earlier sweep); the test confirmed it.
> Affected: main (cd2ce899764e; tested). camel-4.23.x, camel-4.22.x,
> camel-4.18.x and camel-4.14.x put the key into the cache before the store
> accesses the same way (read, not run).
> Duplicate check (2026-10-15): JIRA text "FileIdempotentRepository" (17
> issues, the latest CAMEL-24953 / CAMEL-24954 about the idempotent consumer,
> none about a failing store); GitHub pull requests "FileIdempotentRepository"
> (#22114, camel-atom; nothing related).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
