Federico Mariani created CAMEL-26995:
----------------------------------------

             Summary: camel-core - Aggregate EIP: when a recoverable repository 
throws while confirming a processed aggregated exchange, the recover task sends 
it again
                 Key: CAMEL-26995
                 URL: https://issues.apache.org/jira/browse/CAMEL-26995
             Project: Camel
          Issue Type: Bug
          Components: camel-core
    Affects Versions: 4.18.5, 4.23.0, 4.22.2
            Reporter: Federico Mariani
             Fix For: 4.24.0


With a {{RecoverableAggregationRepository}} ({{JdbcAggregationRepository}} and 
others), the aggregated exchange of a completed group stays in the completed 
store of the repository until {{confirmWithResult}} removes it, once the 
exchange was processed successfully. CAMEL-17613 made the Aggregate EIP retry a 
confirm that returns {{false}} from the recover task, instead of sending the 
exchange again ("If the remove failed, retry it in the RecoverTask, but do not 
redeliver the exchange").

A confirm that *throws* is not handled the same way 
({{core/camel-core-processor/src/main/java/org/apache/camel/processor/aggregate/AggregateProcessor.java}},
 lines at {{camel-4.23.0}}, the code is the same on main):
* {{AggregateOnCompletion.onComplete}} (line 1523) adds the exchange id to 
{{unconfirmedCompleteExchanges}} only when {{confirmWithResult}} returns 
{{false}} (lines 1530-1540). When it throws, this is skipped, the {{finally}} 
(line 1543) removes the id from {{inProgressCompleteExchanges}}, and 
{{UnitOfWorkHelper.doneSynchronization}} logs {{Exception occurred during 
onCompletion. This exception will be ignored.}}
* On its next pass, {{RecoverTask}} finds the id in the completed store, 
neither in progress nor unconfirmed (line 1725), recovers the exchange (line 
1734) and submits it again (line 1794) with {{CamelRedelivered=true}}. The 
group is processed twice.

{{JdbcAggregationRepository.confirmWithResult}} runs its {{DELETE}} in a 
{{TransactionTemplate}}, and throws ({{DataAccessException}} / 
{{TransactionException}}) when the database or the connection is not available 
at that moment. This was found with fault-injection tests against PostgreSQL 
(sessions killed, connection refused while the group was being confirmed). The 
other recoverable repositories (camel-cassandraql, camel-hazelcast, 
camel-infinispan, camel-redis, camel-caffeine, camel-ehcache, 
{{KeyValueAggregationRepository}}) use the default {{confirmWithResult}}, which 
calls {{confirm}}, so they throw whatever their store throws.

{{RecoverTask.retryConfirm}} (line 1812) has the same gap: a retried confirm 
that throws ends the whole recover pass ({{Error during recover task. This 
exception is ignored.}}), so the exchanges after it in the scan are neither 
confirmed nor recovered in that pass, and not at all while that confirm keeps 
failing and comes first in the scan (for example a key whose owner node is not 
available).

h3. Steps to reproduce

An in-memory {{RecoverableAggregationRepository}} that moves a completed group 
to a completed store (like {{JdbcAggregationRepository}}), with a recovery 
interval of 100 ms, whose next {{confirmWithResult}} throws:

{code:java}
from("direct:start")
    .aggregate(header("id"), 
strategy).aggregationRepository(repository).completionSize(2)
    .to("mock:result");
// send A and B with header id=1; the confirm of the aggregated exchange A+B 
throws once
{code}

h3. Actual (4.18.5, 4.22.2, 4.23.0 and main)

{{mock:result}} receives {{A+B}} twice, the second time with 
{{CamelRedelivered=true}} and {{CamelRedeliveryCounter=1}}. The confirm after 
the second delivery succeeds.

The same scenario with a confirm that returns {{false}} (the CAMEL-17613 case) 
delivers {{A+B}} once.

h3. Expected

{{A+B}} is delivered once, and the recover task confirms it again later, as for 
a confirm that returns {{false}}.

h3. Suggested fix

In {{AggregateOnCompletion.onComplete}}, catch the exception of 
{{confirmWithResult}}, log a WARN with the exchange id, and keep the exchange 
as not confirmed, as for {{false}} (add the id to 
{{unconfirmedCompleteExchanges}} and remove its redelivery state). In 
{{RecoverTask.retryConfirm}}, log it and go on with the next exchange of the 
scan, as for {{false}}.

If a confirm throws after the row was deleted, the id stays in 
{{unconfirmedCompleteExchanges}}, as it already does for a {{false}} result 
whose row is gone: one id per such failure, against a duplicate delivery today.

h3. Related

* CAMEL-17613: a confirm that returns {{false}} is retried, not redelivered.
* CAMEL-26739: ids that stay in {{unconfirmedCompleteExchanges}}.
* CAMEL-24943, CAMEL-26642: other cases where the recover task sent an exchange 
twice.

_Claude Code on behalf of Croway_




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to