shashank created CAMEL-27349:
--------------------------------
Summary: camel-core - Idempotent Consumer: an exchange redelivered
by the error handler of a calling route is dropped as a duplicate of itself, so
the message is lost
Key: CAMEL-27349
URL: https://issues.apache.org/jira/browse/CAMEL-27349
Project: Camel
Issue Type: Bug
Components: camel-core
Reporter: shashank
Assignee: shashank
h3. What happens
With the defaults ({{eager=true}}, {{removeOnFailure=true}},
{{completionEager=false}}, {{skipDuplicate=true}}), the Idempotent Consumer
adds the message id when the exchange enters it and registers an on-completion
that removes the id when the exchange is done and failed.
If the exchange fails inside the idempotent consumer and the error handler of a
calling route redelivers it, the same exchange enters the idempotent consumer
again. Its own message id is still in the repository (the on-completion has not
run yet), so {{add}} returns false and the exchange is taken for a duplicate of
itself:
* the block is skipped, so the message is never processed;
* the redelivery counts as successful: the exchange completes without an error
and never reaches the dead letter channel;
* the on-completion then sees a successful exchange and keeps (confirms) the
message id, so every later copy of the message is dropped as a duplicate too.
With {{skipDuplicate(false)}} the redelivered exchange is routed on with
{{CamelDuplicateMessage=true}}, into the duplicate branch of the route.
The error handler documentation describes two ways a calling route redelivers
into a sub route: the sub route uses {{noErrorHandler}} (the documented way to
retry an entire sub route), or the exception escapes a {{doTry}} block of the
sub route, which keeps its own error handler. A {{doCatch}} of the calling
route that sends the failed exchange to the sub route again hits the same
duplicate. {{completionEager(true)}} is not affected (the id is removed as soon
as the block fails), nor is {{eager(false)}}.
h3. How to reproduce
{code:java}
errorHandler(deadLetterChannel("mock:dead").maximumRedeliveries(2).redeliveryDelay(0));
from("direct:start").to("direct:sub");
from("direct:sub").errorHandler(noErrorHandler())
.idempotentConsumer(header("messageId"), new KeyValueIdempotentRepository())
.process(e -> { if (attempts.getAndIncrement() == 0) throw new
IllegalStateException("Forced failure"); })
.to("mock:result")
.end();
{code}
Send one message with {{messageId=1}}. Expected: the first attempt fails, the
redelivery processes the message ({{mock:result}} gets it). Actual:
{{mock:result}} gets nothing, {{mock:dead}} gets nothing, the exchange has no
exception, and the repository keeps {{1}}. When every attempt fails, the
message does not reach {{mock:dead}} either: the first redelivery is skipped as
a duplicate and counts as a success.
Found with a TLA+ model of the Idempotent Consumer, its on-completion and an
in-memory repository (redelivery by a calling route, completionEager,
non-eager, LRU eviction and repository failures), then reproduced against the
real classes.
h3. Fix
When an exchange that added the message id fails in the idempotent consumer
(the exception leaves the block), it is marked as the owner of that id, as long
as its on-completion is still pending on the exchange. If the same exchange
enters the idempotent consumer again, it is processed again under the
on-completion it already registered, instead of being taken for a duplicate.
The same applies when a {{doCatch}} of the calling route sends the failed
exchange to the sub route again. Other exchanges with the same id stay
duplicates. An exchange whose block succeeded and that failed after the
idempotent consumer is still not processed again by the redelivery, and
{{removeOnFailure=false}} keeps the current behaviour (the redelivery is a
duplicate).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)