Federico Mariani created CAMEL-27008:
----------------------------------------

             Summary: camel-core - Aggregate EIP: when a recoverable repository 
throws while confirming an exchange that the recover task moved to the dead 
letter channel, it is moved there again
                 Key: CAMEL-27008
                 URL: https://issues.apache.org/jira/browse/CAMEL-27008
             Project: Camel
          Issue Type: Bug
          Components: camel-core
            Reporter: Federico Mariani
             Fix For: 4.24.0


With a {{RecoverableAggregationRepository}} that has a {{deadLetterUri}} and 
{{maximumRedeliveries}}, the recover task moves an aggregated exchange that 
keeps failing to the dead letter channel, and then confirms it on the 
repository, so that it is not recovered again. That confirm is not guarded 
({{core/camel-core-processor/src/main/java/org/apache/camel/processor/aggregate/AggregateProcessor.java}},
 lines on main at {{21b8ed8e9613}}; the code of this path is the same in 4.22.x 
and 4.23.0):

* {{RecoverTask.run}} sends the exhausted exchange to the dead letter channel 
(line 1907) and then calls {{recoverable.confirm(camelContext, exchangeId)}} 
(line 1921). When that confirm throws, the exception leaves the scan loop and 
is logged by the outer catch ({{Error during recover task. This exception is 
ignored.}}, line 1965).
* The id is not added to {{unconfirmedCompleteExchanges}}, and its redelivery 
state still says it is exhausted. On the next pass the scan returns it again, 
it is neither in progress nor unconfirmed (line 1874), so it is recovered (line 
1883) and moved to the dead letter channel *again*. This repeats for as long as 
the confirm fails.
* The ids after it in the scan are neither recovered nor confirmed in that 
pass, and not at all while that confirm keeps failing and comes first in the 
scan.

This is the dead letter channel counterpart of CAMEL-26995, which made a 
confirm that throws in {{AggregateOnCompletion.onComplete}} and 
{{RecoverTask.retryConfirm}} be handled like a confirm that returns {{false}} 
(CAMEL-17613: "retry it in the RecoverTask, but do not redeliver the 
exchange"). The dead letter path also calls {{confirm}} where the javadoc of 
{{AggregationRepository.confirm}} says that {{confirmWithResult}} is invoked 
instead for a recoverable repository, so a {{false}} result is ignored there, 
and the exchange is moved again if the repository still returns it.

{{JdbcAggregationRepository.confirm}} runs a {{DELETE}} in a 
{{TransactionTemplate}} and throws when the database or the connection is not 
available, as fault-injection tests against PostgreSQL showed for the confirm 
after processing (CAMEL-26995). The other recoverable repositories throw 
whatever their store throws.

h3. Steps to reproduce

An in-memory {{RecoverableAggregationRepository}} that moves a completed group 
to a completed store (like {{JdbcAggregationRepository}}), with a recovery 
interval of 100 ms, {{maximumRedeliveries=2}} and {{deadLetterUri=mock:dead}}:

{code:java}
from("direct:start")
    .aggregate(header("id"), 
strategy).aggregationRepository(repository).completionSize(2)
    .to("mock:result")
    .throwException(new IllegalArgumentException("fails every time"));
// send A and B with header id=0; the confirm of A+B after it was moved to 
mock:dead throws once
{code}

h3. Actual (main)

* The confirm throws once: {{mock:dead}} receives {{A+B}} twice.
* The confirm returns {{false}} once: {{mock:dead}} receives {{A+B}} twice.
* Two failing groups A+B and C+D, the confirm of A+B keeps throwing: 
{{mock:dead}} receives {{A+B}} at every pass (about 100 times in 10 seconds), 
with {{Error during recover task. This exception is ignored.}} each time, and 
C+D is never moved to the dead letter channel nor confirmed.

On 4.18.x, whose {{RecoverTask.run}} does not have the catch of CAMEL-24141, 
the exception of that confirm ends the scheduled recover task instead: no 
exchange is recovered any more until the route is restarted.

h3. Expected

Each exchange is moved to the dead letter channel once. A confirm that throws 
or returns {{false}} is retried by the recover task as a confirm, as in 
{{onComplete}}, and the pass goes on with the other ids.

h3. Suggested fix

After the exchange was moved to the dead letter channel, call 
{{confirmWithResult}}; when it throws (except a {{VirtualMachineError}}, as 
elsewhere in the class since CAMEL-26881) or returns {{false}}, log and add the 
id to {{unconfirmedCompleteExchanges}}, so the next passes call 
{{retryConfirm}} for it. Remove its redelivery state, as {{onComplete}} does.

h3. Related

* CAMEL-26995: the same for the confirm in {{onComplete}} and {{retryConfirm}}.
* CAMEL-17613: a confirm that returns {{false}} is retried, not redelivered.

_Claude Code on behalf of Croway_




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to