Lorenz Quack created QPID-6700:
----------------------------------
Summary: Out of order delivery of messages caused by out of order
message rejection
Key: QPID-6700
URL: https://issues.apache.org/jira/browse/QPID-6700
Project: Qpid
Issue Type: Bug
Components: Java Broker
Reporter: Lorenz Quack
Analysis of a failing systest
(MessageConsumerCloseTest#testPrefetchedMessagesReleasedOnConsumerClose)
revealed the following an issue where during consumer close prefetched messages
can be rejected out of order resulting in out of order delivery to another
consumer.
* The main thread calls consumer.close which locks the messageDeliveryLock
before sending Cancel and waiting synchronously for a CancelOk.
* During that time the Dispatcher pops a message from the pre-dispatch queue
and calls dispatchMessage which will spin in a loop trying to lock the
messageDeliveryLock. (Delivery Tag 25 in the attached log)
* Once the CancelOk arrives on the IO-Recv thread the
BasicCancelOkMethodHandler calls session.confirmConsumerCancelled which ends up
rejecting the remaining messages on the pre-dispatch queue. (Delivery Tag 26 in
the attached log)
* Only after this happens is the messageDeliveryLock released by the main
thread so that the dispatcher can reject the message it has popped from the
pre-dispatch queue and was in the process of dispatching. (Delivery Tag 25)
Possible fixes:
* acquire dispatcherLock on consumer#close, session#close, and connection#close
or
* be more conservative where we acquire the messageDeliveryLock in the
dispatcher (hold it while we fetch message from pre-dispatch queue and handle
it). Taking this approach could possibly open up the possiblity of merging
dispatcherLock and messageDeliveryLock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
