allthingssecurity opened a new pull request, #30130: URL: https://github.com/apache/camel/pull/30130
# Description [CAMEL-26956](https://issues.apache.org/jira/browse/CAMEL-26956), part of CAMEL-26610. A deterministic simulation test (camel-test-dst) of the camel-azure-servicebus consumer in peek-lock mode, in the `dst` package of its tests, in the same form as the SQS world: `ServiceBusConsumerDstTest` (1000 seeds), `ServiceBusWorld` and `FakeServiceBus`. Test only; no main code changes. What the world models: the real consumer (`ServiceBusConsumer` and its lock renewer) and Camel's routing run on the virtual-time loop; only Service Bus is faked, behind the processor and receiver clients of a Mockito mock of `ServiceBusClientFactory` set on the endpoint (`setServiceBusClientFactory`), the clients themselves being Mockito mocks of the final SDK classes. The fake is written from the Service Bus pages on message locks and settlement and on dead-letter queues, and from azure-messaging-servicebus 7.17, and quotes them: a received message is locked for the lock duration of the queue; when the lock expires or the message is abandoned it goes back for redelivery with a higher delivery count, and past MaxDeliveryCount it goes to the dead-letter queue; complete, abandon and dead-letter fail once the lock is lost; a renewal takes a while and fails when the lock is lost by the time the server handles it; and "Settling a message after you close the receiver throws an exception ". In 7.17 the processor of a queue runs the V2 stack by default, where `stop()` closes the processor and its receiver, so the fake throws on a settlement after `stop()` or `close()`. A settlement of a delivery that was settled already, and a renewal through a closed client or of a settled delivery, are recorded as misuse. The fake also renews the lock of a message while its `processMessage` call runs, as the processor client does: the builder Javadoc says a processor client "is configured with auto-completion and auto-lock renewal capabilities" and that `maxAutoLockRenewDuration` "controls how long the background renewal task runs" (the factory passes the endpoint's value), and in 7.17 (javap of 7.17.16) the V2 `MessagePump` begins the renewal before it calls `processMessage` and disposes it when the call returns; `LockRenewalOperation` renews when min(half of the time left, 10 seconds) is left. The renewal ends when `maxAutoLockRenewDuration` has passed, when the call returns, or when `stop()` or `close()` disposed the receiver. With the asynchronous route a routed message's call returns at once, so this matters only for a message the consumer holds in `processMessage` without routing it. The route has an asynchronous step (a downstream call), so `processMessage` returns at once and only the lock renewer of the consumer keeps the locks of the routed messages. The seed picks the lock duration of the queue, the max delivery count, `maxConcurrentCalls`, `maxAutoLockRenewDuration` (60 or 300 seconds), `enableDeadLettering`, the latency of the renewals and the fault rates, and injects failed and slow renewals, failed settlements, failed processing, processing longer than the lock duration, and route restarts while messages are processed. The operator stops the route as `DefaultShutdownStrategy` does (`deferShutdown`, a check every second for the exchanges in flight and pending, up to the timeout of the strategy, then `stopRoute`), with the wait of the strategy as tasks of the operator. A plain `stopRoute` would not work once the consumer holds a message in `processMessage` until it stops: on the one stack of the harness that call runs above the wait of the strategy, and the stop below it cannot go on. When the timeout passes first, the consumer is forced to stop, as the strategy does, and the exchanges still in flight fail (stopping the consumer stops the processor of its route, which then rejects them); their settlements find the client closed, by design. One difference from the real strategy, written in the Javadoc: after the forced stop the world calls `stopRoute`, which is a second graceful shutdown that waits for those exchanges, so the route stop and the restart come later than they would. Invariants: no lock is lost, and no message is delivered again, while the consumer holds the message (a routed delivery whose exchange is not done; a message held in `processMessage` without being routed is not counted, the processor client renews its lock), unless a renewal failed, the route was stopped while it was processed or `maxAutoLockRenewDuration` passed; the message of a completed exchange is completed, unless its settlement failed or was excused that way; a message is completed only after its exchange completed, and abandoned or dead-lettered only after it failed (or was in flight when a stop timed out); once the faults cleared every message is completed or dead-lettered; the consumer settles each delivery at most once and renews only what it holds, through an open client, and a delivered message is routed or settled by the time `processMessage` returns, unless the route was stopped and not started again (a consumer that returned early from `processMessage` would otherw ise only show up as messages dead-lettered after MaxDeliveryCount, which the last invariant accepts; with such a mutant 190 of 200 seeds fail). Two behaviours of the consumer are accepted by the default set and counted with a property, so reviewers can judge them: - After one renewal error the consumer stops renewing that message, also when the error was transient, so the lock is lost later and the message is delivered again while it is still processed. With `-Ddst.servicebus.strictRenew=true` 28 of 1000 seeds on main fail. Filed as CAMEL-26969, fix in its own PR. - `deferShutdown` calls `stop()` on the processor client, which in SDK 7.17 closes the receiver, so an exchange that completes during the graceful stop (between that `stop()` and the `close()` of the consumer's stop) cannot settle its message, and the message is delivered again after its lock expires. With `-Ddst.servicebus.strictClose=true`, which counts only those settlements and not the ones after a stop that timed out, 96 of 1000 seeds on main fail (fewer than before because a stop that times out is now forced at the timeout of the strategy, and the settlements after that are excused). Filed as CAMEL-26970, fix in its own PR, which pairs with this one. With either fix merged locally the default set passes 1000 of 1000 seeds, and with the CAMEL-26970 fix `-Ddst.servicebus.strictClose=true` passes as well (5000 of 5000 seeds). That fix holds a message delivered during the graceful stop in `processMessage` until the consumer stops, which is why the world models the processor's lock renewal during the call and stops the route in steps. Limits, written in the world's Javadoc: the processor's lock renewal during `processMessage` has no injected faults or latency (the faults are on the consumer's own renewals). Not explored: sessions, receiveAndDelete mode, a processor client given by the user (the consumer does not renew its locks), a synchronous route, `maxAutoLockRenewDuration=0`, topics. Not modelled: prefetch, deferral, scheduled messages, time to live, duplicate detection, connection loss and the processError callback. Other changes: - `camel-test-dst` as a test dependency of camel-azure-servicebus, and its byte code agent on the surefire command line, as for the other worlds (mockito-core was a test dependency already). - The test log of camel-azure-servicebus goes to a size-capped rolling file (50 MB, one backup), and the route of the world does not log each injected processing failure. - The `@DstWorld` annotation lists the world in the component JSON (and its catalog copy) and in the table of the camel-test-dst page (and its catalog copy, byte-identical). Tests: - On main: 5000 of 5000 seeds pass (14 to 18 ms a seed, depending on the load of the machine). Seeds 8 and 77 replayed in two JVMs give the same trace hashes. - Self-check against a fixed bug: on the code before CAMEL-25701 (4f879148b118 reverted) 76 of 2000 seeds fail, all on "no lock is lost and no message is delivered again while the consumer holds it". - camel-azure-servicebus unit tests: 1075 run (75, plus 1000 seeds), 0 failures, 0 errors, 0 skipped. - With the CAMEL-26970 fix merged locally: default set 1000 of 1000, strictClose 5000 of 5000 (15 to 24 ms a seed, depending on the load of the machine; the earlier version of the world ran about 190 ms a seed there, waiting on a stop it could not reach), camel-azure-servicebus unit tests 1079 run (79, plus 1000 seeds), 0 failures. With the CAMEL-26969 fix merged locally: default set 1000 of 1000. What was not run locally: the ITs (they need Docker) and the full root build. I built and tested offline with the versions cached on my machine: Jackson 2.22.3, commons-lang3 3.21.0 and Guava 33.7.2 (main has 2.23.0, 3.22.0 and 33.8.0-jre). # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26956](https://issues.apache.org/jira/browse/CAMEL-26956). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested camel-test-dst, camel-azure-common and camel-azure-servicebus, including the formatter and import-sort plugins. The generated files touched are the component JSON with its catalog copy, and the table of the camel-test-dst page with its catalog copy (byte-identical). I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
