[ 
https://issues.apache.org/jira/browse/CAMEL-27066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

shashank updated CAMEL-27066:
-----------------------------
    Description: 
Found by the DST world of CAMEL-26956 once the CAMEL-26969 fix was on main.

h3. What happens

The {{LockRenewer}} of {{ServiceBusConsumer}} runs every 10 s (or a second 
later) and renews a message lock when it expires within 20 s. Since CAMEL-26969 
a transient renewal error keeps the entry, so the next run tries again. But 
depending on where the runs fall, the first renewal of a lock can come with 
only 9 to 11 s left, for any lock duration, and the next run then comes after 
the lock expired.

Example with a queue lock duration of 30 s (seed 113 of the world):
* t=37 s: the lock is renewed until 68.9 s.
* t=48 s: nothing to do, the lock expires later than now + 20 s.
* t=59 s: the renewal fails with a transient error.
* t=70 s: the next run renews, but the lock expired at 68.9 s. The renewal 
fails with a lost lock and the message is delivered again while the first 
exchange is still being processed.

The same can happen with longer locks (for example 42 s with 11 s gaps, or 60 s 
with four 10 s gaps in a row). In the world, 11 of 2000 seeds fail this way 
with {{-Ddst.servicebus.strictRenew=true}}.

h3. Fix

Run the lock renewer every 5 s instead of 10 s and keep the 20 s threshold (as 
a constant of its own). When the runs come on time, the first renewal comes 
with more than 14 s left and a retry after a transient error with more than 8 s 
left, without a timer and without more renewal calls when renewals succeed (a 
60 s lock is still renewed about every 42 s). A short bounded retry, or 
renewing within 30 s with 10 s runs, were considered: the first needs a timer 
or a retry in the error callback, the second renews more often (a 60 s lock 
about every 33 s instead of 44) and still failed 9 of 2000 seeds in the world. 
With the fix, the world fails 1 of 2000 with strictRenew (three faults in a 
row). The world then drops strictRenew and excuses a lost lock only after two 
renewal faults in a row.

_Filed with Claude Code on behalf of allthingssecurity._

  was:
Found by the DST world of CAMEL-26956 once the CAMEL-26969 fix was on main.

h3. What happens

The {{LockRenewer}} of {{ServiceBusConsumer}} runs every 10 s (or a second 
later) and renews a message lock when it expires within two intervals. Since 
CAMEL-26969 a transient renewal error keeps the entry, so the next run tries 
again. But the next run comes about 10 s later, and by then the lock may be 
gone.

Example with a queue lock duration of 30 s (seed 113 of the world):
* t=37 s: the lock is renewed until 68.9 s.
* t=48 s: nothing to do, the lock expires later than now + 20 s.
* t=59 s: the renewal fails with a transient error.
* t=70 s: the next run renews, but the lock expired at 68.9 s. The renewal 
fails with a lost lock and the message is delivered again while the first 
exchange is still being processed.

So with a lock duration below about 40 s, one transient error is enough to lose 
the lock. In the world, 11 of 2000 seeds fail this way (all with a 30 s lock 
and a renewal fault rate of 0.05 or 0.2).

h3. Fix

Try a transient failure again soon, while the lock is still held, instead of 
waiting for the next run (a short bounded retry of the renewal), and keep the 
next run as the fallback. The world then runs with 
{{-Ddst.servicebus.strictRenew=true}} as its default, and the excuse for 
renewal faults goes away.

_Filed with Claude Code on behalf of allthingssecurity._


> camel-azure-servicebus - the Camel lock renewer retries a failed renewal only 
> on its next run, which can come after the lock expired
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-27066
>                 URL: https://issues.apache.org/jira/browse/CAMEL-27066
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-azure-servicebus
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Minor
>
> Found by the DST world of CAMEL-26956 once the CAMEL-26969 fix was on main.
> h3. What happens
> The {{LockRenewer}} of {{ServiceBusConsumer}} runs every 10 s (or a second 
> later) and renews a message lock when it expires within 20 s. Since 
> CAMEL-26969 a transient renewal error keeps the entry, so the next run tries 
> again. But depending on where the runs fall, the first renewal of a lock can 
> come with only 9 to 11 s left, for any lock duration, and the next run then 
> comes after the lock expired.
> Example with a queue lock duration of 30 s (seed 113 of the world):
> * t=37 s: the lock is renewed until 68.9 s.
> * t=48 s: nothing to do, the lock expires later than now + 20 s.
> * t=59 s: the renewal fails with a transient error.
> * t=70 s: the next run renews, but the lock expired at 68.9 s. The renewal 
> fails with a lost lock and the message is delivered again while the first 
> exchange is still being processed.
> The same can happen with longer locks (for example 42 s with 11 s gaps, or 60 
> s with four 10 s gaps in a row). In the world, 11 of 2000 seeds fail this way 
> with {{-Ddst.servicebus.strictRenew=true}}.
> h3. Fix
> Run the lock renewer every 5 s instead of 10 s and keep the 20 s threshold 
> (as a constant of its own). When the runs come on time, the first renewal 
> comes with more than 14 s left and a retry after a transient error with more 
> than 8 s left, without a timer and without more renewal calls when renewals 
> succeed (a 60 s lock is still renewed about every 42 s). A short bounded 
> retry, or renewing within 30 s with 10 s runs, were considered: the first 
> needs a timer or a retry in the error callback, the second renews more often 
> (a 60 s lock about every 33 s instead of 44) and still failed 9 of 2000 seeds 
> in the world. With the fix, the world fails 1 of 2000 with strictRenew (three 
> faults in a row). The world then drops strictRenew and excuses a lost lock 
> only after two renewal faults in a row.
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to