Just entered in my radar, today: https://github.com/OptimalBits/bull
It uses redis for persistence. Each queue has name, redis server, redis port. And you can send any message to a named queue And process any message from a named queue, I suppose from any machine that can access redis server. Angel "Java" Lopez @ajlopez On Thu, Dec 26, 2013 at 10:41 AM, Daniel Rinehart <[email protected]>wrote: > At my last company we had good success with coffee-resque backed by Redis > with a little custom coding around it. Each worker was an independent > process that was managed and kept alive with node-foreman and monit. Jobs > added to the queue included a timeout that the worker used to trigger > timeouts along with handling uncaught exceptions. Not a cookie cutter > system, but it kept things simple by gluing together a bunch of known good > modules. > > -- Daniel R. <[email protected]> [http://danielr.neophi.com/] > > > On Wed, Dec 25, 2013 at 9:23 PM, Tim Shnaider <[email protected]> wrote: > >> Hi Node.js experts, >> >> There are many many modules and frameworks able to solve this problem. >> I'm looking for CONCRETE recommendations and guidance for well supported >> active implementations. >> e.g. pubsub, message queues etc >> >> I'm a bit overwhelmed by the options and amount of research testing I'd >> have to do so hoping the community can help me narrow down the choices. >> >> Usual scenario: >> >> * Front end (FE) server receives request >> * FE submits task/job >> * Available worker pulls task or is notifed, executes, returns result >> * Result can be any JSON response >> * FE notified of response/result via Promise/Callback >> >> Objective: >> * Keep FE responsive focused on executing cheap operations to serve FE. >> * Implement architecture where tasks do not execute within a timeout can >> be terminated. >> >> Worker architecture: >> * Scale across multiple servers e.g. I just need to add more physical >> nodes >> * Configurable execution timeout resulting in termination/restart of >> worker process and returning a Timeout style notification >> * Will execute static JS file i.e. NOT dynamic task code >> * Resilient - handles worker process failure elegantly and restart/spawn >> more. >> >> I guess a common implementation will be a Worker Master running on a >> server, >> spawning X number of children that monitor a queue (pull) or are pushed a >> task via sockets/pipes or some other mechanism. >> >> It will be up to the Master on each physical node to monitor child worker >> processes and spawn more as needed. >> >> Thanks, >> Tim >> >> >> >> >> >> >> -- >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/nodejs?hl=en?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "nodejs" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
