Federico Mariani created CAMEL-26995:
----------------------------------------
Summary: camel-core - Aggregate EIP: when a recoverable repository
throws while confirming a processed aggregated exchange, the recover task sends
it again
Key: CAMEL-26995
URL: https://issues.apache.org/jira/browse/CAMEL-26995
Project: Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 4.18.5, 4.23.0, 4.22.2
Reporter: Federico Mariani
Fix For: 4.24.0
With a {{RecoverableAggregationRepository}} ({{JdbcAggregationRepository}} and
others), the aggregated exchange of a completed group stays in the completed
store of the repository until {{confirmWithResult}} removes it, once the
exchange was processed successfully. CAMEL-17613 made the Aggregate EIP retry a
confirm that returns {{false}} from the recover task, instead of sending the
exchange again ("If the remove failed, retry it in the RecoverTask, but do not
redeliver the exchange").
A confirm that *throws* is not handled the same way
({{core/camel-core-processor/src/main/java/org/apache/camel/processor/aggregate/AggregateProcessor.java}},
lines at {{camel-4.23.0}}, the code is the same on main):
* {{AggregateOnCompletion.onComplete}} (line 1523) adds the exchange id to
{{unconfirmedCompleteExchanges}} only when {{confirmWithResult}} returns
{{false}} (lines 1530-1540). When it throws, this is skipped, the {{finally}}
(line 1543) removes the id from {{inProgressCompleteExchanges}}, and
{{UnitOfWorkHelper.doneSynchronization}} logs {{Exception occurred during
onCompletion. This exception will be ignored.}}
* On its next pass, {{RecoverTask}} finds the id in the completed store,
neither in progress nor unconfirmed (line 1725), recovers the exchange (line
1734) and submits it again (line 1794) with {{CamelRedelivered=true}}. The
group is processed twice.
{{JdbcAggregationRepository.confirmWithResult}} runs its {{DELETE}} in a
{{TransactionTemplate}}, and throws ({{DataAccessException}} /
{{TransactionException}}) when the database or the connection is not available
at that moment. This was found with fault-injection tests against PostgreSQL
(sessions killed, connection refused while the group was being confirmed). The
other recoverable repositories (camel-cassandraql, camel-hazelcast,
camel-infinispan, camel-redis, camel-caffeine, camel-ehcache,
{{KeyValueAggregationRepository}}) use the default {{confirmWithResult}}, which
calls {{confirm}}, so they throw whatever their store throws.
{{RecoverTask.retryConfirm}} (line 1812) has the same gap: a retried confirm
that throws ends the whole recover pass ({{Error during recover task. This
exception is ignored.}}), so the exchanges after it in the scan are neither
confirmed nor recovered in that pass, and not at all while that confirm keeps
failing and comes first in the scan (for example a key whose owner node is not
available).
h3. Steps to reproduce
An in-memory {{RecoverableAggregationRepository}} that moves a completed group
to a completed store (like {{JdbcAggregationRepository}}), with a recovery
interval of 100 ms, whose next {{confirmWithResult}} throws:
{code:java}
from("direct:start")
.aggregate(header("id"),
strategy).aggregationRepository(repository).completionSize(2)
.to("mock:result");
// send A and B with header id=1; the confirm of the aggregated exchange A+B
throws once
{code}
h3. Actual (4.18.5, 4.22.2, 4.23.0 and main)
{{mock:result}} receives {{A+B}} twice, the second time with
{{CamelRedelivered=true}} and {{CamelRedeliveryCounter=1}}. The confirm after
the second delivery succeeds.
The same scenario with a confirm that returns {{false}} (the CAMEL-17613 case)
delivers {{A+B}} once.
h3. Expected
{{A+B}} is delivered once, and the recover task confirms it again later, as for
a confirm that returns {{false}}.
h3. Suggested fix
In {{AggregateOnCompletion.onComplete}}, catch the exception of
{{confirmWithResult}}, log a WARN with the exchange id, and keep the exchange
as not confirmed, as for {{false}} (add the id to
{{unconfirmedCompleteExchanges}} and remove its redelivery state). In
{{RecoverTask.retryConfirm}}, log it and go on with the next exchange of the
scan, as for {{false}}.
If a confirm throws after the row was deleted, the id stays in
{{unconfirmedCompleteExchanges}}, as it already does for a {{false}} result
whose row is gone: one id per such failure, against a duplicate delivery today.
h3. Related
* CAMEL-17613: a confirm that returns {{false}} is retried, not redelivered.
* CAMEL-26739: ids that stay in {{unconfirmedCompleteExchanges}}.
* CAMEL-24943, CAMEL-26642: other cases where the recover task sent an exchange
twice.
_Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)