Hi Istvan, I've not yet seen Riak hacked into a queue, and I suspect it's for good reason. Your design sounds like a lot of work just to emulate the functionality that something that like Kafka or RabbitMQ can do out of the box (though I have to admit the "Riak as a Queue" headline on HackerNews would be quite nice).
Out of curiosity, any chance you made a pass at it over the last few weeks? Mark twitter.com/pharkmillups On Wed, Nov 7, 2012 at 11:58 PM, Istvan Soos <[email protected]> wrote: > Hi, > > I'm playing with the idea how Riak could be used to build a pub/sub > messaging on it. I know that there are specialized products for that, > I just find it an interesting problem to solve it with Riak only (+ > not too much client code). > > The example scenario is the following: > - N producers create e.g. "task items", store it in Riak > - M workers/consumers try to complete them and after completion delete > the items from Riak > - workers choose from "one of the oldest" task items, e.g. there is a > close-to-fair scheduling (but no absolute ordering) > - workers try to not work on the same task > > In the first iteration, I'd use: > - secondary index with the actual timestamp to provide ordering > - the workers could use (hash modulo M) decision to work on a given task > - the workers periodically re-sync their status (e.g. number of active > workers) through a different bucket > > I'm not entirely convinced that this is a scalable design, but it > seems good on the design board. Does anyone have a working experience > with similar designs? Any hint that x/y/z could go wrong? > > Thanks, > Istvan > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
