Federico Mariani created CAMEL-24091:
----------------------------------------
Summary: camel-file: completion-time idempotentKey evaluated
against route-mutated headers - failed files never retried (eager) or endless
duplicates (non-eager); regression of CAMEL-21733
Key: CAMEL-24091
URL: https://issues.apache.org/jira/browse/CAMEL-24091
Project: Camel
Issue Type: Bug
Components: camel-ftp, camel-file
Affects Versions: 4.21.0
Reporter: Federico Mariani
Attachments:
FileConsumerIdempotentRollbackChangedFileNameHeaderIssueTest.java
h3. Problem
When {{idempotentKey}} is an expression (the documented
{{${file:name}-${file:size}}} style), the poll-time key is evaluated on a
pristine file exchange ({{dynamic == null}} for scheduled polls). At
completion, {{GenericFileOnCompletion.processStrategyCommit/Rollback}}
(GenericFileOnCompletion.java:119, 162) re-evaluates the key via
{{GenericFileHelper.createDummy(endpoint, exchange, ...)}} - and
{{createDummy}} copies *all headers of the routed exchange over the dummy with
override=true* ({{MessageHelper.copyHeaders(..., true)}}). Any route that
mutates {{CamelFileName}}/{{CamelFileLength}} - the classic
{{setHeader(Exchange.FILE_NAME, ...)}} before a file/ftp producer - makes the
completion-time key differ from the poll-time key.
h3. Failure scenarios
* {{idempotentEager=true}} (default): rollback calls {{remove(wrongKey)}} ->
the eagerly-added real key stays -> a *failed file is never retried* (silent
loss).
* {{idempotentEager=false}}: commit calls {{add(wrongKey)}} -> real key never
added -> with {{noop=true}} the file is *re-consumed on every poll* (endless
duplicates).
* Eager + repos where {{confirm()}} is meaningful (JDBC/Kafka-style
pending-confirm repos): {{confirm(wrongKey)}} leaves the real entry unconfirmed
-> re-consumed after restart.
h3. Related variant (pre-existing)
With {{recursive=true}} + {{preMove}}, the OnCompletion holds the
*post-preMove* file ({{GenericFileConsumer.java:470}}), and
{{GenericFile.changeFileName}} keeps only the last path segment - so
{{${file:name}}}-based keys differ from poll time even without header mutation,
with the same failure modes.
h3. History
Before CAMEL-21733 (f27c9b212768, Camel 4.11) the completion dummy was
{{endpoint.createExchange(file)}} - pure file headers. The enrichment was added
so *dynamic poll* caller headers stay resolvable, but on the completion path
the "dynamic" argument is the fully-routed exchange, overriding the very
{{CamelFile*}} headers the key is computed from.
h3. Suggested fix
Snapshot the poll-time key on the exchange (property) and reuse it at
commit/rollback - this fixes both the header-mutation regression and the
preMove/recursive variant in one place. Alternatively, exclude {{CamelFile*}}
headers from the completion-path enrichment.
h3. Reproducer
Attached failing JUnit test (place in
{{core/camel-core/src/test/java/org/apache/camel/component/file/}}, where the
file component tests live). Route sets {{CamelFileName}} then fails on first
attempt; with the bug the rollback removes the wrong key and the file is never
retried.
----
_This issue was researched and filed by Claude Code on behalf of [~croway]
(GitHub: Croway), as part of a deep code review of camel-file and camel-ftp. A
failing JUnit reproducer is attached; it fails deterministically on current
main (4.22.0-SNAPSHOT)._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
