[
https://issues.apache.org/jira/browse/CAMEL-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-21659 started by Simon Rasmussen.
-----------------------------------------------
> Improve aws sqs TimeoutExtender by introducing deadline concept to messages
> ---------------------------------------------------------------------------
>
> Key: CAMEL-21659
> URL: https://issues.apache.org/jira/browse/CAMEL-21659
> Project: Camel
> Issue Type: Improvement
> Components: camel-aws2-sqs
> Affects Versions: 4.8.3
> Reporter: Simon Rasmussen
> Assignee: Simon Rasmussen
> Priority: Major
> Fix For: 4.8.4, 4.10.0
>
>
> After upgrading to 4.8.3, we started seeing the following warnings in our
> logs, quite often:
> {noformat}
> 2025-01-27T06:54:34.226Z
> WARN 11 --- [X] [TimeoutExtender]
> o.a.c.component.aws2.sqs.Sqs2Consumer ::
> [BatchResultErrorEntry(Id=FBDB3014CD64A96-0000000000093816,
> SenderFault=true, Code=ReceiptHandleIsInvalid, Message=Message does
> not exist or is not available for visibility timeout change)]{noformat}
> The error occurs when an attempt to extend message visibility for a message
> that has already been deleted in SQS.
> In CAMEL-19656, the implementation was switched to using the batched call,
> but failed to implement the proper ignore in case of above error – as the old
> implementation had.
> I also noticed that the current implementation extends ALL messages in the
> queue of non-processed – not just those that are about to expire. This
> happens because the TimeoutExtender, currently will make a copy of all the
> values in the Map, and extend them, in batches.
> In case of 100 messages, this will require 10 requests, even though the last
> messages in the queue might not need to be extended yet.
> If the route is used together with the threads EIP, then the default queue
> size is 1000, which makes above algorithm really inefficient.
> I suggest introducing a concept of a deadline to each entry in the
> TimeoutExtender. The deadline is set to now() plus 80% of the
> visibilityTimeout. When sending requests, only messages that will expire
> before the next execution of the TimeoutExtender will be extended. The
> TimeoutExtender task itself should run more frequently, but will pick up less
> messages each time. I suggest it running every visibilityTimeout/2. In a
> typical configuration, this would result in executing TimeoutExtender every
> 10 seconds.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
