allthingssecurity opened a new pull request, #30130:
URL: https://github.com/apache/camel/pull/30130

   # Description
   
   [CAMEL-26956](https://issues.apache.org/jira/browse/CAMEL-26956), part of 
CAMEL-26610.
   
   A deterministic simulation test (camel-test-dst) of the 
camel-azure-servicebus consumer in peek-lock mode, in the `dst` package of its 
tests, in the same form as the SQS world: `ServiceBusConsumerDstTest` (1000 
seeds), `ServiceBusWorld` and `FakeServiceBus`. Test only; no main code changes.
   
   What the world models: the real consumer (`ServiceBusConsumer` and its lock 
renewer) and Camel's routing run on the virtual-time loop; only Service Bus is 
faked, behind the processor and receiver clients of a Mockito mock of 
`ServiceBusClientFactory` set on the endpoint (`setServiceBusClientFactory`), 
the clients themselves being Mockito mocks of the final SDK classes. The fake 
is written from the Service Bus pages on message locks and settlement and on 
dead-letter queues, and from azure-messaging-servicebus 7.17, and quotes them: 
a received message is locked for the lock duration of the queue; when the lock 
expires or the message is abandoned it goes back for redelivery with a higher 
delivery count, and past MaxDeliveryCount it goes to the dead-letter queue; 
complete, abandon and dead-letter fail once the lock is lost; a renewal takes a 
while and fails when the lock is lost by the time the server handles it; and 
"Settling a message after you close the receiver throws an exception
 ". In 7.17 the processor of a queue runs the V2 stack by default, where 
`stop()` closes the processor and its receiver, so the fake throws on a 
settlement after `stop()` or `close()`. A settlement of a delivery that was 
settled already, and a renewal through a closed client or of a settled 
delivery, are recorded as misuse.
   
   The fake also renews the lock of a message while its `processMessage` call 
runs, as the processor client does: the builder Javadoc says a processor client 
"is configured with auto-completion and auto-lock renewal capabilities" and 
that `maxAutoLockRenewDuration` "controls how long the background renewal task 
runs" (the factory passes the endpoint's value), and in 7.17 (javap of 7.17.16) 
the V2 `MessagePump` begins the renewal before it calls `processMessage` and 
disposes it when the call returns; `LockRenewalOperation` renews when min(half 
of the time left, 10 seconds) is left. The renewal ends when 
`maxAutoLockRenewDuration` has passed, when the call returns, or when `stop()` 
or `close()` disposed the receiver. With the asynchronous route a routed 
message's call returns at once, so this matters only for a message the consumer 
holds in `processMessage` without routing it.
   
   The route has an asynchronous step (a downstream call), so `processMessage` 
returns at once and only the lock renewer of the consumer keeps the locks of 
the routed messages. The seed picks the lock duration of the queue, the max 
delivery count, `maxConcurrentCalls`, `maxAutoLockRenewDuration` (60 or 300 
seconds), `enableDeadLettering`, the latency of the renewals and the fault 
rates, and injects failed and slow renewals, failed settlements, failed 
processing, processing longer than the lock duration, and route restarts while 
messages are processed. The operator stops the route as 
`DefaultShutdownStrategy` does (`deferShutdown`, a check every second for the 
exchanges in flight and pending, up to the timeout of the strategy, then 
`stopRoute`), with the wait of the strategy as tasks of the operator. A plain 
`stopRoute` would not work once the consumer holds a message in 
`processMessage` until it stops: on the one stack of the harness that call runs 
above the wait of the strategy, and
  the stop below it cannot go on. When the timeout passes first, the consumer 
is forced to stop, as the strategy does, and the exchanges still in flight fail 
(stopping the consumer stops the processor of its route, which then rejects 
them); their settlements find the client closed, by design. One difference from 
the real strategy, written in the Javadoc: after the forced stop the world 
calls `stopRoute`, which is a second graceful shutdown that waits for those 
exchanges, so the route stop and the restart come later than they would.
   
   Invariants: no lock is lost, and no message is delivered again, while the 
consumer holds the message (a routed delivery whose exchange is not done; a 
message held in `processMessage` without being routed is not counted, the 
processor client renews its lock), unless a renewal failed, the route was 
stopped while it was processed or `maxAutoLockRenewDuration` passed; the 
message of a completed exchange is completed, unless its settlement failed or 
was excused that way; a message is completed only after its exchange completed, 
and abandoned or dead-lettered only after it failed (or was in flight when a 
stop timed out); once the faults cleared every message is completed or 
dead-lettered; the consumer settles each delivery at most once and renews only 
what it holds, through an open client, and a delivered message is routed or 
settled by the time `processMessage` returns, unless the route was stopped and 
not started again (a consumer that returned early from `processMessage` would 
otherw
 ise only show up as messages dead-lettered after MaxDeliveryCount, which the 
last invariant accepts; with such a mutant 190 of 200 seeds fail).
   
   Two behaviours of the consumer are accepted by the default set and counted 
with a property, so reviewers can judge them:
   - After one renewal error the consumer stops renewing that message, also 
when the error was transient, so the lock is lost later and the message is 
delivered again while it is still processed. With 
`-Ddst.servicebus.strictRenew=true` 28 of 1000 seeds on main fail. Filed as 
CAMEL-26969, fix in its own PR.
   - `deferShutdown` calls `stop()` on the processor client, which in SDK 7.17 
closes the receiver, so an exchange that completes during the graceful stop 
(between that `stop()` and the `close()` of the consumer's stop) cannot settle 
its message, and the message is delivered again after its lock expires. With 
`-Ddst.servicebus.strictClose=true`, which counts only those settlements and 
not the ones after a stop that timed out, 96 of 1000 seeds on main fail (fewer 
than before because a stop that times out is now forced at the timeout of the 
strategy, and the settlements after that are excused). Filed as CAMEL-26970, 
fix in its own PR, which pairs with this one.
   
   With either fix merged locally the default set passes 1000 of 1000 seeds, 
and with the CAMEL-26970 fix `-Ddst.servicebus.strictClose=true` passes as well 
(5000 of 5000 seeds). That fix holds a message delivered during the graceful 
stop in `processMessage` until the consumer stops, which is why the world 
models the processor's lock renewal during the call and stops the route in 
steps.
   
   Limits, written in the world's Javadoc: the processor's lock renewal during 
`processMessage` has no injected faults or latency (the faults are on the 
consumer's own renewals). Not explored: sessions, receiveAndDelete mode, a 
processor client given by the user (the consumer does not renew its locks), a 
synchronous route, `maxAutoLockRenewDuration=0`, topics. Not modelled: 
prefetch, deferral, scheduled messages, time to live, duplicate detection, 
connection loss and the processError callback.
   
   Other changes:
   - `camel-test-dst` as a test dependency of camel-azure-servicebus, and its 
byte code agent on the surefire command line, as for the other worlds 
(mockito-core was a test dependency already).
   - The test log of camel-azure-servicebus goes to a size-capped rolling file 
(50 MB, one backup), and the route of the world does not log each injected 
processing failure.
   - The `@DstWorld` annotation lists the world in the component JSON (and its 
catalog copy) and in the table of the camel-test-dst page (and its catalog 
copy, byte-identical).
   
   Tests:
   - On main: 5000 of 5000 seeds pass (14 to 18 ms a seed, depending on the 
load of the machine). Seeds 8 and 77 replayed in two JVMs give the same trace 
hashes.
   - Self-check against a fixed bug: on the code before CAMEL-25701 
(4f879148b118 reverted) 76 of 2000 seeds fail, all on "no lock is lost and no 
message is delivered again while the consumer holds it".
   - camel-azure-servicebus unit tests: 1075 run (75, plus 1000 seeds), 0 
failures, 0 errors, 0 skipped.
   - With the CAMEL-26970 fix merged locally: default set 1000 of 1000, 
strictClose 5000 of 5000 (15 to 24 ms a seed, depending on the load of the 
machine; the earlier version of the world ran about 190 ms a seed there, 
waiting on a stop it could not reach), camel-azure-servicebus unit tests 1079 
run (79, plus 1000 seeds), 0 failures. With the CAMEL-26969 fix merged locally: 
default set 1000 of 1000.
   
   What was not run locally: the ITs (they need Docker) and the full root 
build. I built and tested offline with the versions cached on my machine: 
Jackson 2.22.3, commons-lang3 3.21.0 and Guava 33.7.2 (main has 2.23.0, 3.22.0 
and 33.8.0-jre).
   
   # Target
   
   - [x] I checked that the commit is targeting the correct branch (Camel 4 
uses the `main` branch)
   
   # Tracking
   - [x] If this is a large change, bug fix, or code improvement, I checked 
there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for 
the change (usually before you start working on it): 
[CAMEL-26956](https://issues.apache.org/jira/browse/CAMEL-26956).
   
   # Apache Camel coding standards and style
   
   - [x] I checked that each commit in the pull request has a meaningful 
subject line and body.
   - [ ] I have run `mvn clean install -DskipTests` locally from root folder 
and I have committed all auto-generated changes.
     (I built and tested camel-test-dst, camel-azure-common and 
camel-azure-servicebus, including the formatter and import-sort plugins. The 
generated files touched are the component JSON with its catalog copy, and the 
table of the camel-test-dst page with its catalog copy (byte-identical). I did 
not run the full root build.)
   
   # AI-assisted contributions
   
   - [x] If this PR includes AI-generated code, commits have proper 
co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR 
description identifies the AI tool used.
     This PR was prepared with Claude Code (Claude Opus 5.5). The commit 
carries a `Co-Authored-By` trailer.
   
   _Claude Code on behalf of allthingssecurity_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to