Michael, Interesting read but it requires that the clients have a birds eye view of the hash ring which IMO is an over kill for what we want to achieve. I think this approach is better done if the server is doing the spreading. I would gladly settle for X_REQ/X_REP to achieve that but unfortunately the dealer/router model does not fit the bill for this particular app. I do have control over the queue where the tasks are popped and i think this is the best place to spread the load evenly and the hash ring approach might just be what we need to achieve it. I really appreciate the input.
Joegen On 01/25/2012 10:51 AM, Michel Pelletier wrote: > If you use a hash ring approach, you can hash the message to one > unique worker who does the pop, an the rest who fail to hash to the > message go off and do the fail logic. That way there's good random > distribution and no delays. Here's a blog on the approach: > > http://www.lexemetech.com/2007/11/consistent-hashing.html > > -Michel > > On Tue, Jan 24, 2012 at 6:41 PM, Joegen Baclor<[email protected]> wrote: >> Fair enough. Indeed it can be done by a virtual random delay or >> back-off mechanism if a client exceeds a certain threshold. Thanks for >> the input martin, that's all I need. >> >> On 01/25/2012 10:35 AM, Martin Sustrik wrote: >>> On 01/25/2012 11:30 AM, Michel Pelletier wrote: >>>> If I follow what you're asking, you want the numbers in your first >>>> email to have a more even distribution than what they have now, where >>>> no one subscriber gets the message "first", so to speak. I'm not sure >>>> it makes sense for 0mq to try to make any guarantees on the time of >>>> arrival. >>> It cannot do that. Exact arrival time depends on your network, jitter in >>> the OS and similar. >>> >>>> Messages could be going out same or different transports and >>>> interfaces. Maybe a better solution for you would be for your workers >>>> to "roll the dice" to see who will pop first. >>> Yes. You can implement that by introducing a random delay between >>> receiving the pub/sub message and popping the queue. >>> >>> Martin >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
