My typical method is to randomly add jobs to servers (unless for some reason I need things to be consumed in a certain order) and then have multiple consumers each pointed at a different server.
My tubes all see constant action however so my setup kind of necessitates that. --chad On Jul 15, 2011, at 12:59 PM, Alister Bulman <[email protected]> wrote: On Friday, July 15, 2011 12:19:12 PM UTC+1, Andrew Betts wrote: > > So, if that's the case, how would a worker know that a job was available on > a server other than the one it sent the reserve to? Is it simply a matter > of running enough workers that there's always at least one watching each > beanstalkd server? Is that the generally recommended way of implementing a > pool class (if I were to write one in PHP)? Would it be better to have each > worker connected to all (or at least several) beanstalk servers, and if so, > any suggestions for how a single threaded worker should alternate between > servers? > > And are there any PHP client libraries I'm not aware of that already have > this kind of functionality? > There's an open issue for multi-server support in Pheanstalk, but no movement on it yet, https://github.com/pda/pheanstalk/issues/17 As you'd pretty much said, the two options are to either connect, with a short or no-timeout [reserve(0), in Pheanstalk] to each server in turn, or just be running enough workers and trust that probability will get to them all eventually. The former means that you could end up in quite a tight loop, so I'd tend towards the latter, as I'll usually be running a dozen or more workers. Since my own queues can also empty out, I'd rather have a few seconds reserve-with-timeout call. Getting stats for the number of ready-jobs from the server is easy enough, so you'll be able to see if the queues are going down or not anyway. You could also do both as well, choose a couple of beanstalkd servers at random and reserve from one, then try the next, rinse and repeat. Alister -- I presented http://abulman.co.uk/presentations/Beanstalkd-2010-05-06/ at PHP London, May 2010 -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To view this discussion on the web visit https://groups.google.com/d/msg/beanstalk-talk/-/WZeCNWFau0wJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en. -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
