[
https://issues.apache.org/jira/browse/CAMEL-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-26995 started by Federico Mariani.
------------------------------------------------
> camel-core - Aggregate EIP: when a recoverable repository throws while
> confirming a processed aggregated exchange, the recover task sends it again
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26995
> URL: https://issues.apache.org/jira/browse/CAMEL-26995
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 4.18.5, 4.22.2, 4.23.0
> Reporter: Federico Mariani
> Assignee: Federico Mariani
> Priority: Minor
> Fix For: 4.24.0
>
>
> With a {{RecoverableAggregationRepository}} ({{JdbcAggregationRepository}}
> and others), the aggregated exchange of a completed group stays in the
> completed store of the repository until {{confirmWithResult}} removes it,
> once the exchange was processed successfully. CAMEL-17613 made the Aggregate
> EIP retry a confirm that returns {{false}} from the recover task, instead of
> sending the exchange again ("If the remove failed, retry it in the
> RecoverTask, but do not redeliver the exchange").
> A confirm that *throws* is not handled the same way
> ({{core/camel-core-processor/src/main/java/org/apache/camel/processor/aggregate/AggregateProcessor.java}},
> lines at {{camel-4.23.0}}, the code is the same on main):
> * {{AggregateOnCompletion.onComplete}} (line 1523) adds the exchange id to
> {{unconfirmedCompleteExchanges}} only when {{confirmWithResult}} returns
> {{false}} (lines 1530-1540). When it throws, this is skipped, the {{finally}}
> (line 1543) removes the id from {{inProgressCompleteExchanges}}, and
> {{UnitOfWorkHelper.doneSynchronization}} logs {{Exception occurred during
> onCompletion. This exception will be ignored.}}
> * On its next pass, {{RecoverTask}} finds the id in the completed store,
> neither in progress nor unconfirmed (line 1725), recovers the exchange (line
> 1734) and submits it again (line 1794) with {{CamelRedelivered=true}}. The
> group is processed twice.
> {{JdbcAggregationRepository.confirmWithResult}} runs its {{DELETE}} in a
> {{TransactionTemplate}}, and throws ({{DataAccessException}} /
> {{TransactionException}}) when the database or the connection is not
> available at that moment. This was found with fault-injection tests against
> PostgreSQL (sessions killed, connection refused while the group was being
> confirmed). The other recoverable repositories (camel-cassandraql,
> camel-hazelcast, camel-infinispan, camel-redis, camel-caffeine,
> camel-ehcache, {{KeyValueAggregationRepository}}) use the default
> {{confirmWithResult}}, which calls {{confirm}}, so they throw whatever their
> store throws.
> {{RecoverTask.retryConfirm}} (line 1812) has the same gap: a retried confirm
> that throws ends the whole recover pass ({{Error during recover task. This
> exception is ignored.}}), so the exchanges after it in the scan are neither
> confirmed nor recovered in that pass, and not at all while that confirm keeps
> failing and comes first in the scan (for example a key whose owner node is
> not available).
> h3. Steps to reproduce
> An in-memory {{RecoverableAggregationRepository}} that moves a completed
> group to a completed store (like {{JdbcAggregationRepository}}), with a
> recovery interval of 100 ms, whose next {{confirmWithResult}} throws:
> {code:java}
> from("direct:start")
> .aggregate(header("id"),
> strategy).aggregationRepository(repository).completionSize(2)
> .to("mock:result");
> // send A and B with header id=1; the confirm of the aggregated exchange A+B
> throws once
> {code}
> h3. Actual (4.18.5, 4.22.2, 4.23.0 and main)
> {{mock:result}} receives {{A+B}} twice, the second time with
> {{CamelRedelivered=true}} and {{CamelRedeliveryCounter=1}}. The confirm after
> the second delivery succeeds.
> The same scenario with a confirm that returns {{false}} (the CAMEL-17613
> case) delivers {{A+B}} once.
> h3. Expected
> {{A+B}} is delivered once, and the recover task confirms it again later, as
> for a confirm that returns {{false}}.
> h3. Suggested fix
> In {{AggregateOnCompletion.onComplete}}, catch the exception of
> {{confirmWithResult}}, log a WARN with the exchange id, and keep the exchange
> as not confirmed, as for {{false}} (add the id to
> {{unconfirmedCompleteExchanges}} and remove its redelivery state). In
> {{RecoverTask.retryConfirm}}, log it and go on with the next exchange of the
> scan, as for {{false}}.
> If a confirm throws after the row was deleted, the id stays in
> {{unconfirmedCompleteExchanges}}, as it already does for a {{false}} result
> whose row is gone: one id per such failure, against a duplicate delivery
> today.
> h3. Related
> * CAMEL-17613: a confirm that returns {{false}} is retried, not redelivered.
> * CAMEL-26739: ids that stay in {{unconfirmedCompleteExchanges}}.
> * CAMEL-24943, CAMEL-26642: other cases where the recover task sent an
> exchange twice.
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)