shashank created CAMEL-25701:
--------------------------------
Summary: camel-azure-servicebus - the Camel lock renewer waits for
its next run when a message lock expires within 10 seconds, but runs only every
10 to 11 seconds, so locks of long-running exchanges are lost and the messages
are delivered again
Key: CAMEL-25701
URL: https://issues.apache.org/jira/browse/CAMEL-25701
Project: Camel
Issue Type: Bug
Components: camel-azure-servicebus
Reporter: shashank
Assignee: shashank
Since CAMEL-23937 the Service Bus consumer renews the message locks itself in
{{PEEK_LOCK}} mode (the default, with the default {{maxAutoLockRenewDuration}}
of 5 minutes, sessions disabled): the SDK's own renewal stops when Camel's
callback returns, which is immediately for an asynchronous route.
{{ServiceBusConsumer}} schedules a {{LockRenewer}} with
{{PeriodTaskScheduler.schedulePeriodTask(lockRenewer, 10000)}}, and the renewer
renews a lock when {{now + 10 seconds}} is after the lock's expiry
({{LOCK_RENEW_INTERVAL_SECONDS}}, line 347 on main d46d51c24550).
The period task scheduler is driven by a 1-second timer
({{TimerListenerManager}}), and {{PeriodicTaskWrapper}} runs the task when more
than 10000 ms have passed since the last run, so two runs are 10 or 11 seconds
apart. The window in which the renewer acts is 10 seconds long. When the lock
expires between 10 and 11 seconds after a run, that run leaves it alone ("not
yet within 10 seconds") and the next run comes after the lock has expired, or
so close to the expiry that the renewal request arrives too late. The renewal
then fails with a lost lock, the entry is dropped (WARN "Failed to renew
lock"), Service Bus delivers the message again to this or another consumer
while the first exchange is still being processed, and the final {{complete()}}
of the first exchange fails.
A small exhaustive model of the timing (every sequence of 6 run gaps of 10 or
11 seconds, lock expiry at every 100 ms phase) loses the lock in 6.0% of the
cases with the current 10-second margin (11% when the runs come every 11
seconds, 1% when they come exactly every 10 seconds: an expiry that falls on a
run), and in none with a margin of two intervals. Which gap the scheduler
produces depends on sub-millisecond timer jitter. An exchange that is processed
for 5 minutes with the default 60-second lock duration crosses about 5 lock
periods, so with 6% per period roughly one such exchange in four is delivered
twice.
Trigger: default options ({{PEEK_LOCK}}, {{maxAutoLockRenewDuration}} 5
minutes), no sessions, an asynchronous route (the case CAMEL-23937 is about),
and an exchange that takes longer than the lock duration of the queue or
subscription minus 10 seconds.
h3. Reproduction
New test in {{ServiceBusConsumerTest}} (Mockito, no Service Bus): the renewer
is captured from the scheduler together with its period; a message whose lock
expires 2 seconds after the earliest possible next run ({{period + 2 s}}) is
tracked, and the renewer runs once. Waiting for the next run would leave at
most 2 seconds, or none. On main the renewer does not renew:
{noformat}
Wanted but not invoked:
renewalClient.renewMessageLock(message);
{noformat}
A second new test (a lock that expires in 60 seconds is not renewed) passes on
main and with the fix; it guards against renewing too early.
h3. Proposed fix
Renew when the lock expires within two intervals (20 seconds) instead of one:
every lock is then renewed by a run that comes at least about 9 seconds before
it expires. A lock is still renewed about once per lock duration (somewhat
earlier than before); with a lock duration of 20 seconds or less it is renewed
at every run. No option changes.
camel-azure-servicebus unit tests with the fix: 79 run, 0 failures, 8 skipped
(integration tests). Not run: a real namespace.
How it was found: comparing the lock renewal timing with the visibility
extender of camel-aws2-sqs (which renews when the deadline falls before the
next expected run) during a sweep of acknowledge and lock timing; the test and
the timing model confirmed it.
Affected: main (tested). camel-4.23.x and camel-4.22.x (line 336) and
camel-4.18.x (line 310) have the same renewer (read, not run).
Duplicate check (2026-10-16): JIRA text "renewMessageLock" / "lock renewal"
with component camel-azure-servicebus (CAMEL-23937, CAMEL-23938 session lock
renewal, resolved Not A Problem, its PR #25001 closed unmerged; none about the
margin); GitHub pull requests "servicebus lock renew" (#24511 merged, #25001
closed unmerged, sessions).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
