Hi,
While working on https://issues.apache.org/jira/browse/PROTON-200, I've been
able to "hang" a messenger server by doing the following simple test:
In one thread I create a message receiver by subscribing to to
"amqp://~0.0.0.0:12345", and simply consuming messages like so:
while True:
server.recv(10)
while server.incoming:
server.get(msg)
....
In another thread, I produce a couple of messages, each with a different "path"
given to the message's address. For example:
msg = Message()
msg.address="amqp://0.0.0.0:12345/XXX"
....
client.put(msg)
client.send()
sleep(1)
msg = Message()
msg.address="amqp://0.0.0.0:12345/YYY"
....
client.put(msg)
client.send()
sleep(1)
msg = Message()
msg.address="amqp://0.0.0.0:12345/ZZZ"
....
client.put(msg)
client.send()
This ends up causing the server loop to hang in send.recv() after receiving the
first message (to amqp://0.0.0.0:12345/XXX). No other messages are retrieved.
The problem is related to PROTON-200, but goes a bit further. It turns out
that for each client message that contains a different path in its address, a
new amqp 1.0 link is created "under the covers" by messenger. And this is
fine. However, each new link requires the receiver (server) to grant it some
credit before a message can be transferred over that link.
Credit is provided by the server application via the "server.recv(10)" call -
this call allows up to 10 messages to be received by granting 10 credits. The
granted credits are evenly distributed across all _known_ links, which seems
like the correct thing to do.
However, the problem occurs when new links are created after the credit has
been doled out. From the example above, when server.recv(10) is invoked, only
the first link has been created. Thus, it gets all the credit. This prevents
later links from being granted any credit. So in the example, the first link
consumes all the credit, doesn't send any more messages, and starves the
remaining links.
I think that this particular example could be solved by having messenger
re-balance credits when new links are created - perhaps by issuing drain as
described in the spec. However this isn't a complete solution. Even with
clever credit re-balancing, if the number of links become greater than the
available credit, a messenger server could hang.
So how does the application determine the proper amount of credit to provide,
given that it doesn't have any control over the link population?
Or, am I just doing it wrong :) ?
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
