On Thu, Oct 6, 2011 at 12:05 PM, Martin Sustrik <sust...@250bpm.com> wrote: > On 10/06/2011 01:02 PM, Martin Sustrik wrote: > >> There are two subproblems to this AFAICS: >> >> 1. How do we communicate the "agreed upon message rate" throughout the >> distribution tree? >> >> 2. How do we notify the applications that they are not adhering the >> message rate contract? > > Ah. One more issue: > > 3. The network itself takes part in the contract, ie. it should be able to > transfer the messages at the agreed upon rate. Who are we going to notify if > it's not keeping up with the rate? Network admin presumably... But how > exactly? > > Martin >
For point 1, that depends on the question of where the message rate comes from - do you have the rate controlled at the publisher (as with PGM i guess), with the clients responsibility to keep up? That results in something pretty similar to the current setup, but with the addition of some sort of 'expected rate' message that could be propagated to leaves of the distribution tree. Or do you have clients advertise the rates at which they consume, and push that responsibility back up? That was roughly where I was going with my initial thought, but it is effectively just setting an window size which would then need a flow of what are basically ACKs back up for every message to add sending credit, which seems like a lot of needless traffic even if you aggregate at each node. For point 2, I think the behavior of blocking on HWM would be better in a 'rate exceeded' cases for publishers. For consumers, I'm not sure how you could signal that they were consuming at too slow a rate, unless you had explicit HWM hit messages - which itself gets back to the problems that we discussed back at the unconf in brussels. For point 3, the basic problem doesn't seem dissimilar to the TCP congestion issue - you agree a maximum rate for flow control (window size, which is really receiver controlled), but then actually have to scale the window size up to that maximum during the communication in order to test what the network can actually handle - if there's packet loss, you scale back. That has the advantage that the network is treated as a dumb resource - which would probably be the right general behavior for PUB/SUB as well. This would need a method of adjusting the rate throughout the distribution tree dynamically though! I guess, if you had a separate nack channel (as in some of Pieter's guide patterns), you could use somewhat similar logic: Publishers have a publishing channel, a NACK channel, and a predefined minimum rate, maximum rate, a rate increment, and clock interval Publishers start publishing on the minimum rate After each clock interval passes, publishers double their rate, until the rate reaches maximum Consumer read messages New consumers read from the first message they receive - they do not request old messages with the NACK channel If a consumer does not get contiguous sequence numbers in message, it sends a NACK to the publisher When the publisher receives a NACK, it halves the message rate After a clock interval with no NACK, having previously received one, it increases the rate by one rate increment per clock interval NACKs could be sent up the chain to allow centrally adjusting rate. This could also be built easily on top of existing zmq pubsub. Ian _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
