Hi Mark,
I can give an outline of how the Java Broker distributes messages
between subscriptions... I'm not familiar with anything the Python
client may do...
In general, when there is no backlog in the queue, the Java Broker
will round-robin between subscriptions which have available credit.
If a subscription runs out of credit then it is marked as suspended.
When such a subscription gets credit again, or when a new
subscription
is added to the queue then the queue will attempt to send as many
messages as it can to this new (or unsuspended) subscription. (To
be
absolutely accurate it attempts to send up to 10 messages, then
yields
- but schedules another attempt to send 10 messages, and so on).
I'm not sure if the above totally explains what you are seeing -
certainly I don't see why the subscriptions should fail to keep up
with the publisher (until you kill the first subscriber) in the way
you describe.
If you could provide a simple example that shows the same behaviour,
that would be fantastic... In the meantime I will have a more
thorough dive into the broker code to see if I can spot anything
obvious.
Hope this helps,
Rob
2010/1/27 mARK bLOORE <[email protected]>:
I am using the java broker 0.5 and the python client. I have a
queue
with one publisher and (usually) three subscribers.
The publisher sends messages at a fairly constant rate.
The subscribers take messages in a single thread at an unlimited
rate,
put them in a buffer, and ack the messages immediately. If the
buffer
fills then they block before the ack.
Normally all the subscribers get messages at about the same rate,
and
the queue's message count is mostly zero. If a subscriber starts
to
block then the message count may rise, and when it gets to the
tens of
thousands I reduce the publication rate. But an odd situation
appears: The blocked subscriber takes messages into its buffer as
fast as it takes them out, but the other two subscribers get
messages
at only one third of the rate that the publisher is adding them.
That
first subscriber is running many times faster.
If I add a fourth subscriber it gets messages at one quarter the
publication rate, and the other two start getting messages at that
rate too. If I kill the fourth subscriber rates return to what
they
were.
If I kill the first subscriber then the other two start taking
messages very fast, and the backlog in the queue quickly
disappears.
It seems as if the broker has earmarked the backlogged messages for
that one subscriber, and won't deliver them to any other unless
that
one goes away. Could that be the case? Note that a subscriber may
have at most one unacked message.
I'm afraid I can't abstract any reasonable amount of code to
display.
I'm not sure that would help anyway.
--
mARK bLOORE <[email protected]>
---
------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
---
------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
