shashank created CAMEL-25759:
--------------------------------
Summary: camel-aws2-eventbridge - a consumer with queueUrl adds a
new target for the same queue at every start, so two consumers sharing the
queue, or a restart after a crash, put every event on the queue twice
Key: CAMEL-25759
URL: https://issues.apache.org/jira/browse/CAMEL-25759
Project: Camel
Issue Type: Bug
Components: camel-aws2
Reporter: shashank
Assignee: shashank
The EventBridge consumer (since 4.19) wires an SQS queue as a target of the
rule and polls the queue. {{EventbridgeConsumer.setupConsumerResources}} (main
e13303ef1ef7, line 188) gives the target a new random id at every start:
{code:java}
targetId = "camel-eb-" + UUID.randomUUID().toString().substring(0, 8);
...
ebClient.putTargets(PutTargetsRequest.builder().rule(ruleName).eventBusName(...).targets(target).build());
{code}
and {{cleanupConsumerResources}} (lines 210-227) removes that target on stop.
With an auto-created queue each consumer has its own queue, so this is fine.
With a queue given by the user ({{queueUrl}}, the documented "User-Provided
Queue" setup; it is also the only way to let several application instances
share the events of a rule, since with auto-created queues every instance gets
its own copy of every event) the rule gets one target per consumer for the
*same* queue:
# two consumers of the queue run at the same time (two instances of the
application, or two routes): the rule has two targets with the queue ARN.
EventBridge sends a matching event to every target of the rule, so each event
is put on the queue twice and routed twice;
# a consumer is not stopped cleanly (JVM killed, container OOM, node lost): its
target stays on the rule. The next start adds a second target for the queue,
and from then on every event is put on the queue twice, with no error and
nothing in the log. Each such restart adds one more copy, until the rule
reaches the EventBridge limit of 5 targets and the consumer fails to start.
In addition, the target is removed when the consumer stops, so a user-owned
queue does not collect the events of the rule while the application is
restarted or redeployed, and with two consumers the first one that stops
removes its own target only, so the duplicates last as long as both run.
*Fix*: the target of a user-provided queue gets an id derived from the queue
ARN ({{camel-eb-}} + 32 hex characters of its SHA-256). PutTargets replaces a
target with the same id, so every start and every consumer of the queue updates
the one target. That target is kept when the consumer stops (other consumers
may share the queue, and the queue keeps collecting events while no consumer
runs). An auto-created queue keeps its random target id and its clean-up
(target removed, queue deleted with {{deleteQueueOnShutdown=true}}). The
component page and the 4.24 upgrade guide say so; a target left by an earlier
version (id {{camel-eb-}} + 8 characters) has to be removed by hand.
*Design choice to review*: keeping the target of a user-provided queue on stop
reverses what CAMEL-23207 did (the component page said "On shutdown the target
is removed", and {{testUserProvidedQueueNotDeletedOnShutdown}} verified the
removal). The two changes go together: with a shared id but removal on stop,
the first consumer that stops would remove the target that the others still
use. Not changed: a target that the user added to the rule for the same queue
(with an id of their own, for example from infrastructure as code) is still a
second target, as on main.
*Tests*: new {{EventbridgeConsumerUserQueueTargetTest}} (4 tests): the mocked
EventBridge client keeps the rule as a map of target id to ARN, with the
PutTargets (replace by id) and RemoveTargets semantics.
{{twoConsumersSharingAQueueAddOneTarget}} and
{{restartAfterACrashAddsNoSecondTarget}} fail on main ({{expected: <1> but was:
<2>}}, two {{camel-eb-xxxxxxxx}} targets for the user queue);
{{userQueueKeepsCollectingEventsWhileTheConsumerIsStopped}} fails on main
(target removed); {{autoCreatedQueueTargetIsStillRemovedOnStop}} passes on main
and with the fix (the auto-created path is unchanged).
{{EventbridgeConsumerTest.testUserProvidedQueueNotDeletedOnShutdown}} pinned
the removal of the user queue's target and now checks that the target is kept.
camel-aws2-eventbridge unit tests: 30 run, 0 failures (the LocalStack ITs need
Docker and were not run).
*How it was found*: a TLA+ model of the targets of the rule (consumers sharing
one user queue that start, stop or crash; each published event is put on the
queue once per target). TLC finds a violation of "no event is put on the queue
twice" in 4 steps with two consumers (start, start, publish) and with one
consumer (start, crash, start, publish); with the fix the property, "an event
published while a consumer runs reaches the queue" and "once a consumer
started, every later event reaches the queue" hold for 2 and 3 consumers. The
JUnit tests above are the evidence; the EventBridge semantics (every target is
invoked; PutTargets with an existing id updates that target) are from the AWS
API reference and were not run against AWS.
*Affected*: main, camel-4.23.x, camel-4.22.x and camel-4.19.x (the same lines;
read, not run). camel-4.18.x has no consumer.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
