Here is a test app. You'll need java 7 and ant to run it. Just clone the repo and type ant as long as you have a beanstalk on localhost:11300
https://github.com/mattcross/beanstalk-perf It adds 3000 tubes, loads them up with 600 jobs each. Then unloads them. It is 2-3x slower than it is on o1.8. On Thursday, June 20, 2013 12:12:58 AM UTC-4, Keith Rarick wrote: > > Thanks for getting in touch separately over email. > As we discussed, a test program that demonstrates > the slowdown will be most helpful. Once the problem > is reproducible, it's much easier to fix! :) > > On Mon, Jun 17, 2013 at 1:54 PM, <[email protected] <javascript:>> > wrote: > > I am prototyping a queueing service for my company that utilizes a > couple > > thousand tubes. I have a test case where I run 3000 tubes with 1200 > jobs > > each in them. Those tubes are serviced by a distributor that takes work > and > > funnels it to a couple of much larger work queues where the real work > > happens. I have 3 clients with 10 threads each processing these larger > > tubes and sending results into yet another tube. > > > > I use delay in the many feeder tubes. I do not use delay in the large > work > > queues. All priorities are the same. > > > > I was having some sporadic crashes due to connection issues in 1.8 so I > > upgraded to 1.9. Since then the crashing has gone away but my test case > > takes 3x longer to run and CPU is maxed out the entire time. With 1.8 > CPU > > would spike during the initial load of the 3000 queues X 1200 jobs but > then > > would settle down to very little during the time the queues were being > > serviced. > > > > In terms of real numbers, my system used to process 600k jobs through > the > > entire system in about 5 minutes. It is now taking over 15. > > > > I am testing on OSX and my clients are on the same machine as > beanstalkd. > > > > Does anyone have any advice or help to offer? > > > > Much appreciated. > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "beanstalk-talk" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > To post to this group, send email to > > [email protected]<javascript:>. > > > Visit this group at http://groups.google.com/group/beanstalk-talk. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/beanstalk-talk. For more options, visit https://groups.google.com/groups/opt_out.
