shashank created CAMEL-25945:
--------------------------------
Summary: camel-azure-storage-queue - the consumer deletes the
message named by the headers at the end of the route, so a forwarded message,
or a route that removes the Camel headers, stays in the queue and is consumed
again and again
Key: CAMEL-25945
URL: https://issues.apache.org/jira/browse/CAMEL-25945
Project: Camel
Issue Type: Bug
Components: camel-azure
Reporter: shashank
Assignee: shashank
{{QueueConsumer}} deletes a message once its exchange completes
({{processCommit}}, main 9f839f4bcc9d). It does so with
{{QueueOperations.deleteMessage(exchange)}}, which reads the message id and the
pop receipt from the {{CamelAzureStorageQueueMessageId}} and
{{CamelAzureStorageQueuePopReceipt}} headers as they are when the route ends,
not as the consumer set them. So:
* a route that forwards the message to another queue,
{{from("azure-storage-queue://account/a").to("azure-storage-queue://account/b")}}:
the producer ({{sendMessage}}) sets both headers to the id and pop receipt of
the message it just added to queue b ({{QueueOperationResponse.buildResponse}}
copies the {{SendMessageResult}} of {{QueueClient.sendMessageWithResponse}}
into them). The consumer then calls Delete Message on queue a with the id of
the message in queue b. Queue Storage answers 404 MessageNotFound, the SDK
throws {{QueueStorageException}}, and {{processCommit}} logs it as "This
exception is ignored";
* a route that removes the Camel headers before it calls another system
({{removeHeaders("Camel*")}}): {{deleteMessage}} throws
{{IllegalArgumentException: Message ID must be specified}}, which
{{processCommit}} does not catch either.
In both cases the received message stays in queue a. It becomes visible again
when its visibility timeout ends (30 seconds by default) and is consumed again,
every time, until its time-to-live ends (7 days by default). With the
forwarding route, each round adds one more copy of the message to queue b.
*Fix*: as CAMEL-16092 did, the consumer keeps the id and pop receipt it
received the message with (taken in {{processBatch}} before the exchange is
routed) and deletes that message. A route that updates the message
({{operation=updateMessage}}, which gives the message a new pop receipt and
makes the old one invalid) sets the new receipt in the header, along with the
same message id. That receipt is used, so this case works as it does today. The
delete uses the endpoint's {{timeout}} option and no longer takes the
producer's {{CamelAzureStorageQueueTimeout}} header.
*Tests*: new {{QueueConsumerDeleteMessageTest}}. Its fake {{QueueClient}}
answers like the Queue Storage REST API: Put Message returns the new id and pop
receipt, Update Message returns a new pop receipt and invalidates the old one,
and Delete Message throws {{QueueStorageException}} with 404 MessageNotFound
for an unknown id and 400 PopReceiptMismatch for a stale receipt. Two of its
tests fail on main:
* {{forwardedMessageIsDeletedFromTheSourceQueue}}: {{Argument(s) are different!
Wanted: deleteMessageWithResponse("a-1", "a-1-receipt-1", ...)}}, but the
actual call was {{deleteMessageWithResponse("b-1", ...)}};
* {{messageIsDeletedWhenTheRouteRemovesTheCamelHeaders}}: {{Wanted but not
invoked: deleteMessageWithResponse(...)}}.
{{updatedMessageIsDeletedWithItsNewPopReceipt}} is a control that passes on
main and with the fix.
camel-azure-storage-queue unit tests with the fix: 14 run, 0 failures.
*How it was found*: library-contract mining. I listed what the Azure Queue SDK
calls return on each path (the {{SendMessageResult}} of Put Message, the new
receipt of Update Message, the 404/400 of Delete Message), then checked the
consumer's commit against those results.
*Affected*: main, camel-4.23.x and camel-4.22.x, since CAMEL-24158
(f6707ee43184). That change removed the save and restore of the message id, pop
receipt and timeout headers around {{processCommit}}, which CAMEL-16092 had
added for this kind of header override. The removal was so that a route that
updates the message keeps its new pop receipt. camel-4.18.x and camel-4.14.x
still restore the received values, so forwarding works there, but a route that
updates the message deletes it with the stale receipt. The fix covers both
cases. I read the code on those branches but did not run the tests there.
Not changed: {{camel-aws2-sqs}} deletes with the {{CamelAwsSqsReceiptHandle}}
header in the same way. Its producer does not set that header, so only
{{removeHeaders("Camel*")}} makes it miss, and SQS stops redelivering after the
queue's {{maxReceiveCount}}. A separate follow-up if wanted.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
