I've a few concerns about my understanding of the real meaning of the
various limits on queue content.
First, let me apologize that my questions need to be interpreted in the
context of the 0.18 release (upgrades here happen slowly, if at all).
Please tell me where I've got it wrong below:
1. Limits on the number of messages in a queue.
I presume that --max-queue-count (qpid.max_count argument in the queue
properties from QMF) works as a limit against the msgDepth statistic from
QMF
msgDepth is always equivalent to (msgTotalEnqueues - msgTotalDequeues).
If a message comes in when msgDepth is already (at least) qpid.max_count,
then the sender receives an exception, the message isn't queued, and the
broker continues to run normally (even if it is part of a cluster).
If --max-queue-count is not specified when a queue is created, then
qpid.max_count doesn't show up in the arguments property, and msgDepth is
unconstrained.
2. Limits on the number of bytes in a queue
I presume that --max-queue-size (qpid.max_size argument in the queue
properties) works as a limit against the byteDepth statistic.
byteDepth is always equivalent to (byteTotalEnqueues - byteTotalDequeues),
even for durable queues where the message body may only exist in the file,
and contrary to some documentation of --mas-queue-size that implies that it
only applies to the in RAM portion of queued messages.
If a message comes in when byteDepth is already (at least) qpid.max_size,
then the sender receives an exception, the message isn't queued, and the
broker continues to run normally (even if it is part of a cluster).
If --max-queue-size is not specified when a queue is created, then
qpid.max_size doesn't show up in the arguments property, and byteDepth is
unconstrained.
3. Limits on the persistence file. (You will find some questions here, and
probably lots of misconceptions.)
--file-count and --file-size (qpid.file_count and qpid.file_size in the QMF
argument property), along with an apparently fixed "page" size of 64kiB,
determine the disk space available to persist this queue. Each of
file_count files is file_size pages.
Are these files pre-allocated, or do they expand to these limits as
messages get persisted?
Messages are not perfectly packed in these files, so the number of bytes of
message that fills this space may be less than the total bytes of disk used.
Is this difference (last paragraph) because of individual message
alignments, or because a message can't be split across files in the set, or
because consumed messages taken from the middle of a file may have that
space re-used by a smaller message, or some combination?
These files only exist for queues marked durable. Specifying --file-count
and --file-size for a non-durable queue is ignored. Is this restriction
implemented in the broker, in qpid-config, or both?
If I add a durable queue (with qpid-config) but don't specify --file-count
and --file-size, I still get qpid.file_count = 8 and qpid.file_size = 12,
as a default size.
I guess that this default sizing is implemented in qpid-config, because a
home grown tool here manages to make queues marked durable, but without
qpid.file_count or qpid.file_size in the arguments property. Do these
queues have persistence space, or are they effectively not durable? If the
broker provides space for this case, how much?
I'm guessing that the file space doesn't directly limit some combination of
statistics, since, for example (bytePersistEnqueues - bytePersistDequeues)
may be somewhat short of the total disk size when the disk space, due to
imperfect packing, becomes full.
When the disk space fills, and the broker is part of a cluster, it shuts
down, rather than refusing the message with an exception to the sender,
over concerns about correct replication to the other cluster members. Or
does this only happen when a replication operation over-fills the file
space?
I see some non-durable queues with non-zero values for
{msg,bytes}Persist{En,De}queues. Is this because messages marked
persistent (durable) is what is actually measured by these parameters, even
when the queue is non-durable, and the messages aren't actually persisted?
Thanks in advance for any clue stick whacking.
Bill
