This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 83ad74ffe10b CAMEL-27066: camel-azure-servicebus - the Camel lock
renewer retries a failed renewal only on its next run, which can come after the
lock expired (#30252)
83ad74ffe10b is described below
commit 83ad74ffe10b8bddfaabf8c23bdbebc963a94a13
Author: allthingssecurity <[email protected]>
AuthorDate: Tue Nov 17 14:30:36 2026 +0530
CAMEL-27066: camel-azure-servicebus - the Camel lock renewer retries a
failed renewal only on its next run, which can come after the lock expired
(#30252)
…a failed renewal only on its next run, which can come after the lock
expired
Since CAMEL-26969 the lock renewer of ServiceBusConsumer keeps an entry
after a transient renewal
error, so its next run renews the lock again. The renewer ran every 10
seconds or a second later
and renewed a lock that expired within 20 seconds. Depending on where the
runs fell, the first
renewal of a lock could come with only 9 to 11 seconds left, for any lock
duration, and when it
failed the next run could come after the lock expired. Service Bus then
delivered the message
again while its exchange was still running.
The renewer now runs every 5 seconds (or a second later) and still renews a
lock that expires
within 20 seconds (now a constant of its own). When the runs come on time,
the first renewal
comes with more than 14 seconds left, a retry after a transient error with
more than 8 seconds
left, and a second retry with more than 2 seconds left. The margins are
smaller when the shared
timer thread of the period task scheduler is late, by the time of the call
to Service Bus, and by
any clock skew between the client and the server.
Why this and not renewing earlier or retrying in the error callback:
- renewing when the lock expires within three intervals (30 seconds) also
survives one error, but
it renews more: a 60 second lock at every third run instead of every
fourth (about every 33
seconds instead of 44), locks of about 31 to 41 seconds at every run
instead of every other
run, and even short-lived messages on 30 second locks;
- with the shorter interval the renewal calls stay as they were (a 60
second lock about every 42
to 45 seconds, a 30 second lock about every 12 to 15 seconds), and the
renewer still makes one
call per lock per run, from the same task, with no new thread, timer or
Reactor delay;
- the cost is a run of the renewer every 5 seconds (a scan of the exchanges
in flight, no call to
Service Bus unless a lock is due) and a retry 5 or 6 seconds after a
failed renewal, sooner than
the back-off of about 10 seconds that Service Bus asks for after a
ServerBusy error.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../component/azure/servicebus/ServiceBusConsumer.java | 12 +++++++-----
.../azure/servicebus/ServiceBusConsumerTest.java | 17 ++++++++++++-----
2 files changed, 19 insertions(+), 10 deletions(-)
diff --git
a/components/camel-azure/camel-azure-servicebus/src/main/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumer.java
b/components/camel-azure/camel-azure-servicebus/src/main/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumer.java
index edc241df0828..a629a11b222e 100644
---
a/components/camel-azure/camel-azure-servicebus/src/main/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumer.java
+++
b/components/camel-azure/camel-azure-servicebus/src/main/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumer.java
@@ -56,7 +56,11 @@ import org.slf4j.LoggerFactory;
public class ServiceBusConsumer extends DefaultConsumer implements
ShutdownAware {
private static final Logger LOG =
LoggerFactory.getLogger(ServiceBusConsumer.class);
- private static final int LOCK_RENEW_INTERVAL_SECONDS = 10;
+ // the lock renewer runs every LOCK_RENEW_INTERVAL_SECONDS (or a second
later) and renews a lock that expires
+ // within LOCK_RENEW_BEFORE_EXPIRY_SECONDS, so its first renewal comes
with more than 14 seconds left, and a
+ // transient failure of it is retried by the next run with more than 8
seconds left (when the runs come on time)
+ private static final int LOCK_RENEW_INTERVAL_SECONDS = 5;
+ private static final int LOCK_RENEW_BEFORE_EXPIRY_SECONDS = 20;
// Azure recommends putting the exception type in the dead-letter reason
and its stack trace in the description,
// but both are length-limited: the .NET and JavaScript SDKs cap them at
4096 (MaxDeadLetterReasonLength, applied
// to the description as well), and Azure separately warns that a stack
trace here can push a message past the
@@ -389,11 +393,9 @@ public class ServiceBusConsumer extends DefaultConsumer
implements ShutdownAware
continue;
}
- // renew if the lock expires before the run after the next
one: the period task scheduler runs this
- // task every LOCK_RENEW_INTERVAL_SECONDS or a second later,
so waiting for the next run when the lock
- // expires within one interval can be too late, and the
renewal itself is a call to Service Bus
+ // renew if the lock expires within
LOCK_RENEW_BEFORE_EXPIRY_SECONDS (several runs before it expires)
Instant lockExpiry = entry.lockedUntil.toInstant();
- if (now.plusSeconds(2L *
LOCK_RENEW_INTERVAL_SECONDS).isAfter(lockExpiry)) {
+ if
(now.plusSeconds(LOCK_RENEW_BEFORE_EXPIRY_SECONDS).isAfter(lockExpiry)) {
client.renewMessageLock(entry.message)
.subscribe(
newLockedUntil -> {
diff --git
a/components/camel-azure/camel-azure-servicebus/src/test/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumerTest.java
b/components/camel-azure/camel-azure-servicebus/src/test/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumerTest.java
index 6a57138b418b..848d73b3ce3c 100644
---
a/components/camel-azure/camel-azure-servicebus/src/test/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumerTest.java
+++
b/components/camel-azure/camel-azure-servicebus/src/test/java/org/apache/camel/component/azure/servicebus/ServiceBusConsumerTest.java
@@ -732,12 +732,10 @@ public class ServiceBusConsumerTest {
}
@Test
- void lockRenewerRenewsAgainAfterATransientError() throws Exception {
+ void lockRenewerRetriesATransientErrorBeforeTheLockExpires() throws
Exception {
when(configuration.getServiceBusReceiveMode()).thenReturn(ServiceBusReceiveMode.PEEK_LOCK);
when(configuration.getMaxAutoLockRenewDuration()).thenReturn(300000L);
when(messageContext.getMessage()).thenReturn(message);
- // the lock expires before the next runs of the renewer
-
when(message.getLockedUntil()).thenReturn(OffsetDateTime.now().plusSeconds(15));
// as the SDK reports a busy server: a transient error, which can be
retried
ServiceBusException busy = new ServiceBusException(
new AmqpException(true, AmqpErrorCondition.SERVER_BUSY_ERROR,
"server busy", null),
@@ -749,7 +747,15 @@ public class ServiceBusConsumerTest {
consumer.doStart();
ArgumentCaptor<Runnable> taskCaptor = ArgumentCaptor.captor();
-
verify(periodTaskScheduler).schedulePeriodTask(taskCaptor.capture(), anyLong());
+ ArgumentCaptor<Long> periodCaptor = ArgumentCaptor.captor();
+
verify(periodTaskScheduler).schedulePeriodTask(taskCaptor.capture(),
periodCaptor.capture());
+ // two runs are at most the period plus a second apart (the period
task scheduler ticks once per second).
+ // A lock that expires 2 seconds after two such gaps must be
renewed now: left to the next run, its first
+ // renewal could come with one gap and 2 seconds left, and a
transient failure of it would be retried with
+ // about 2 seconds left, too little for the call to Service Bus
+ long maxGapMillis = periodCaptor.getValue() + 1000;
+ when(message.getLockedUntil())
+ .thenReturn(OffsetDateTime.now().plus(Duration.ofMillis(2
* maxGapMillis + 2000)));
processMessageCaptor.getValue().accept(messageContext);
taskCaptor.getValue().run();
@@ -795,7 +801,8 @@ public class ServiceBusConsumerTest {
when(configuration.getServiceBusReceiveMode()).thenReturn(ServiceBusReceiveMode.PEEK_LOCK);
when(configuration.getMaxAutoLockRenewDuration()).thenReturn(300000L);
when(messageContext.getMessage()).thenReturn(message);
-
when(message.getLockedUntil()).thenReturn(OffsetDateTime.now().plusMinutes(1));
+ // the renewer renews a lock that expires within 20 seconds: this one
has two seconds more
+
when(message.getLockedUntil()).thenReturn(OffsetDateTime.now().plusSeconds(22));
try (ServiceBusConsumer consumer = new ServiceBusConsumer(endpoint,
processor)) {
consumer.doStart();