[
https://issues.apache.org/jira/browse/CAMEL-26970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-26970:
--------------------------------
Fix Version/s: 4.24.0
> camel-azure-servicebus - an exchange that completes during a graceful stop
> cannot settle its message: deferShutdown closes the receiver (SDK V2 stack)
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26970
> URL: https://issues.apache.org/jira/browse/CAMEL-26970
> Project: Camel
> Issue Type: Bug
> Components: camel-azure-servicebus
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> Found by the Service Bus DST world of CAMEL-26956 (fork branch
> {{camel-azure-servicebus-dst-world-v2}}, PR opens with the 11-16 batch): its
> default set accepts this case, {{-Ddst.servicebus.strictClose=true}} fails
> 601 of 1000 seeds. Confirmed outside the DST harness against the SDK classes
> (javap of azure-messaging-servicebus 7.17.16) and with a plain unit test
> (below).
> h3. What happens
> CAMEL-23260 made {{ServiceBusConsumer}} {{ShutdownAware}} so that exchanges
> in flight at a graceful stop can still settle their message. Its
> {{deferShutdown}} calls {{client.stop()}} with the comment "stop accepting
> new messages but keep the connection open so that in-flight exchanges can
> still complete/abandon messages". With the SDK version Camel uses that is not
> what {{stop()}} does:
> * {{ServiceBusProcessorClient.stop()}} calls {{ServiceBusProcessor.stop()}}
> when the processor runs on the V2 stack ({{processorV2 != null}});
> * the V2 stack is the default for a non-session processor:
> {{ServiceBusClientBuilder.V2StackSupport.isNonSessionAsyncReceiveEnabled}} is
> true unless {{com.azure.messaging.servicebus.nonSession.asyncReceive.v2}}
> opts out;
> * {{ServiceBusProcessor.stop()}} is {{close()}}, which disposes the
> {{RollingMessagePump}}; the pump is a {{Mono.using}} whose cleanup closes its
> {{ServiceBusReceiverAsyncClient}};
> * {{ServiceBusReceiverAsyncClient.updateDisposition}} on a disposed receiver
> returns {{IllegalStateException}} "Cannot perform operation '...' on a
> disposed receiver".
> So an exchange that completes after {{deferShutdown}} gets that exception
> from {{messageContext.complete()}} (logged as "Exception occurred during
> onCompletion"), the message is not settled, and Service Bus delivers it
> again: the graceful stop processes it twice. A failed exchange cannot abandon
> or dead-letter its message either. This is the log of CAMEL-23260 itself
> ("Cannot perform operation 'abandoned' on a disposed receiver"), which the
> deferral did not cure on the V2 stack.
> h3. Expected
> The intent written in {{deferShutdown}} and in CAMEL-23260: during a graceful
> stop the consumer stops taking new messages and the exchanges in flight still
> settle theirs, before the stop closes the client. Service Bus ("Settle
> messages before you close the receiver or connection", dead-letter queue
> page): "Settling a message after you close the receiver throws an exception,
> and the message isn't settled." and "If you close the receiver or its
> connection before you settle a message (complete, abandon, defer, or
> dead-letter), the settlement doesn't reach the service, so the message stays
> locked until the lock expires. The service then redelivers the message, which
> increases its delivery count."
> h3. Reproduction
> New
> {{ServiceBusConsumerTest.gracefulStopKeepsTheClientOpenUntilTheExchangesInFlightAreSettled}},
> with a processor client mock that behaves as the 7.17 V2 client (stop() and
> close() dispose the receiver; a settlement after that throws the
> IllegalStateException above): a message is in flight, {{deferShutdown}} runs,
> the exchange completes. On main {{complete()}} throws; expected the message
> completed and the client closed only by the stop.
> h3. Fix
> In peek-lock mode {{deferShutdown}} no longer stops the processor client; it
> marks the consumer as stopping. A message the client delivers meanwhile is
> held in {{processMessage}}, neither routed nor settled, until the consumer
> stops; as the client delivers the next message only when {{processMessage}}
> returns, at most one message per concurrent call is held
> ({{maxConcurrentCalls}}, per session with sessions). The stop closes the
> client and then releases them; Service Bus delivers a held message again once
> its lock expires (its delivery count goes up by one, as for the exchanges in
> flight on main). The trade is a few messages delivered again later without
> having been routed, instead of every exchange in flight at the stop being
> routed twice. In receive-and-delete mode nothing needs settling and
> {{deferShutdown}} still stops the client. Branch
> {{camel-azure-servicebus-settle-during-graceful-stop}}.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)