On Tue, Jan 13, 2009 at 6:37 PM, noog_com <[email protected]> wrote: > I am looking into utilizing beanstalkd as our distributed message > queue system for noog.com. So far, we have been tremendously > impressed with the simplicity and power of this tool.
Thanks! And be sure to let us know about other things you think could be improved upon, or if you hit any trouble setting things up. > Hopefully this isn't a redundant feed; however, I was wondering if > beanstalkd has plans to support server selection through hashing (like > memcached)? What I mean is, if I have 3 servers running beanstalkd > and I want to deliver a message into the tube called "tube_a", from > what I gather, the worker script will connect to each of the three > servers looking for jobs on "tube_a." Is this a correct assumption? Typically, each worker asks one server at a time. This works out well if you have more workers than servers (which is typical). > So if I had 100 beanstalkd servers, then it would have to look through > all 100 servers to find a job under "tube_a?" Most current workers will pick one at random and ask it. A fully asynchronous worker would probably ask all of them. But why would you need 100 beanstalkd servers? > If the beanstalk > server list were hashed with the key "tube_a" then, for example, > server 1 would always receive the jobs for "tube_a" (again as memcache > does). That way the worker would immediately know to look at only > server 1 for "tube_a" jobs.....should be much faster...no? I don't know if it would be much faster; I'm skeptical. It probably depends on how many workers and how many servers there are. It also will depend on how many different tubes you have -- it's considered good practice to have as few as possible. You'd need to be careful to handle the case when a server goes down. The workers should be able to gracefully move to another server. > I was thinking about expanding the current beanstalk PHP client > library to do this in order to speed up the job retrieval. Has anyone > already looked in this? What are your thoughts? Give it a shot. If you can demonstrate that it's faster, I'm sure the client library maintainers will be interested in adopting your technique. kr --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en -~----------~----~----~----~------~----~------~--~---
