[
https://issues.apache.org/jira/browse/CAMEL-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-25737:
--------------------------------
Fix Version/s: 4.25.0
(was: 4.24.0)
> camel-aws2-sqs - extendMessageVisibility with a visibilityTimeout below 2
> seconds delivers messages again while they are processed
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25737
> URL: https://issues.apache.org/jira/browse/CAMEL-25737
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-sqs
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 4.25.0
>
>
> With {{extendMessageVisibility=true}} the consumer's timeout extender runs
> every {{max(1, visibilityTimeout / 2)}} whole seconds, and it extends a
> message only once 80 % of its visibility timeout has passed. That leaves this
> much time between an extension and the moment the message becomes visible
> again:
> ||visibilityTimeout||extender period||left before expiry||
> |1|1 s|*0 s* (always late)|
> |2|1 s|1 s|
> |3|1 s|1 s|
> |4|2 s|2 s|
> |5|2 s|1 s|
> |6|3 s|3 s|
> |8|4 s|4 s|
> |10|5 s|5 s|
> With {{visibilityTimeout=1}} every extension after the first arrives exactly
> when the message becomes visible again, so SQS answers {{MessageNotInflight}}
> and another consumer receives the message while the first one still processes
> it. A seeded simulation of the consumer against a fake SQS queue showed this
> in 876 of 2000 runs on main with 2 or 3 concurrent consumers. Values from 2
> to 9 work, but with a margin of only 1 to 2 seconds, which network latency, a
> GC pause or a busy extender (one thread, batches of 10) can use up.
> Also, {{visibilityTimeout=0}} (or negative) passes the existing check in
> {{Sqs2Component}}, and the consumer then silently does not create the
> extender ({{Sqs2Consumer}}, {{visibilityTimeout > 0}}), so
> {{extendMessageVisibility}} is ignored.
> Proposed:
> # Fail at startup when {{extendMessageVisibility=true}} and
> {{visibilityTimeout < 2}}, next to the existing check that
> {{visibilityTimeout}} is set.
> # Log a WARN at startup when {{extendMessageVisibility=true}} and
> {{visibilityTimeout < 10}}: the extender leaves only a 1 to 2 second margin.
> # Document on both options that a visibility timeout of 10 seconds or more is
> recommended with {{extendMessageVisibility}}, and add a note to the 4.24
> upgrade guide for the new startup failure.
> Out of scope (possible follow-up): scheduling the extender in milliseconds
> ({{visibilityTimeout * 1000 / 2}}) would remove the rounding and give a 50 %
> margin for every value, but it changes the timing for all users.
> _Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
