On 04/27/2011 11:32 PM, Craig Forbes wrote:
We have been using durable queues (using the persistent store module)
and the limit-policy ring and max-queue-size to limit the size of the
queues. The queues are not operating as rings as expected. The broker
closes the clients connection with "Unexpected exception: Enqueue
capacity threshold exceeded on queue" when submitting to the
persistent queue before the max queue size is hit.
It appears that the --max-queue-size setting for the ring this
specifies the depth of the message contents (i.e. not including the
headers) but the persistent store holds the content and the headers so
despite the total message content being well below the
--max-queue-size value the persistent store is filling up.
It works as a ring when the --max-queue-size is set small enough but
knowing what that limit should be is not easily determined since the
header size is not know and in our case the size of the message
content is variable.
Is my analysis correct? I could not find any documentation clarifying this.
Yes. However it is not simply that the store counts the headers. The
store sizing needs to take into account other factors such as padding,
dequeue records, storage specific fields etc.
It seems to me that the max-queue-size should specify the total size
(header+body) not just the size of of the message contents. The makes
sense both for the persistent store and when using in-memory queues.
I don't disagree... the one slight complication is deciding exactly how
the size of headers should be computed. The size of the payload is
unambiguous which was why it was used in the first instance.
Below are the details of my test setup.
-Craig
Test process -- this was tested using 0.6, 0.8 and 0.10(with
persistent store code from trunk)
Creating the queue with the following command (So the persistent store
is 1MB (4*4*64K) and the queue limit is set to 75% of that)
qpid-config add queue testq --durable --file-size=4 --file-count=4
--max-queue-size=786432 --limit-policy=ring
Send messages with a 100 byte content until I get the threshold
exceeded message. qpid-tool reports the following for the queue:
msgTotalEnqueues 2021
msgTotalDequeues 0
msgPersistEnqueues 2021
msgPersistDequeues 0
msgDepth 2021
byteDepth 202100
byteTotalEnqueues 202100
byteTotalDequeues 0
bytePersistEnqueues 202100
bytePersistDequeues 0
byteDepth and bytePersistEnqueues are both well below the max queue
size of 786432
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
