Thanks for getting in touch separately over email. As we discussed, a test program that demonstrates the slowdown will be most helpful. Once the problem is reproducible, it's much easier to fix! :)
On Mon, Jun 17, 2013 at 1:54 PM, <[email protected]> wrote: > I am prototyping a queueing service for my company that utilizes a couple > thousand tubes. I have a test case where I run 3000 tubes with 1200 jobs > each in them. Those tubes are serviced by a distributor that takes work and > funnels it to a couple of much larger work queues where the real work > happens. I have 3 clients with 10 threads each processing these larger > tubes and sending results into yet another tube. > > I use delay in the many feeder tubes. I do not use delay in the large work > queues. All priorities are the same. > > I was having some sporadic crashes due to connection issues in 1.8 so I > upgraded to 1.9. Since then the crashing has gone away but my test case > takes 3x longer to run and CPU is maxed out the entire time. With 1.8 CPU > would spike during the initial load of the 3000 queues X 1200 jobs but then > would settle down to very little during the time the queues were being > serviced. > > In terms of real numbers, my system used to process 600k jobs through the > entire system in about 5 minutes. It is now taking over 15. > > I am testing on OSX and my clients are on the same machine as beanstalkd. > > Does anyone have any advice or help to offer? > > Much appreciated. > > -- > You received this message because you are subscribed to the Google Groups > "beanstalk-talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/beanstalk-talk. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/beanstalk-talk. For more options, visit https://groups.google.com/groups/opt_out.
