|
Comment Added :
qpid :
Re: Producer flow control
Producer flow control commented on by Rafael H. Schloming (Jul 28, 2008). Comment:The "slow broker" scenario isn't actually an issue. A slow broker will read messages off of the TCP connection slower than a fast producer will write them. This will cause the TCP window to fill up, and cause TCP writes to block at the producer end of the connection. As long as the producer's client doesn't buffer writes indefinitely, this will eventually cause the producer's send message routine to block, thereby flow controlling the producer to exactly the rate that the slow broker can read data off the connection. The problem we're trying to solve is really about producers and consumers. The simplest example is a single producer sending messages through a queue to a single consumer. If the producer sends 100 messages/second, and the consumer processes 50 messages/second, the queue will inevitably grow, and regardless of how fast/slow the broker is, it will eventually swap itself to death if there is no limit on the queue size. In general there may be many producers sending messages to many consumers on any given queue or topic, however the basic issue is the same. The producers need to somehow get feedback about the rate at which messages sent to a given queue or topic are consumed. It may be tempting to try to apply option (3) from above (pause the connection indefinitely), however that is not a safe thing to do. Given that you need to read acks from the connection in order to dequeue messages, it's easy to construct a scenario where two clients passing messages back and forth would deadlock if the two queues they're using both fill up at the same time. |
Unsubscribe or edit your notifications preferences
