Hi, We've also experienced some performance issues with beanstalk 1.9. I've created a similar test program using a C client. You can check out the benchmark here: https://github.com/swehner/bsperf The results are similar:
=== 1.9 === Using port 11300, threads 20, queues 1500, socksperthread 75 each using 666 jobs Total time: 149.771615 s 6676.832589/s vs. === 1.9 reverting the adaptive epoll commit (1d191ba26b20f402cc8e8ee3a7f7b0de9f1ff78c) === Using port 11300, threads 20, queues 1500, socksperthread 75 each using 666 jobs Total time: 71.081487 s 14068.360725/s Hope this helps, Stefan On Tuesday, July 16, 2013 5:10:26 AM UTC+2, schmichael wrote: > > While I don't doubt there's been a slowdown, and your benchmarks are > useful, the Trendrr client has some glaring bugs and strange code that make > it less than an ideal test client. For example when reading a response from > beanstalk it has the following strange loop: > > > https://github.com/dustismo/TrendrrBeanstalk/blob/master/src/com/trendrr/beanstalk/BeanstalkConnection.java#L101-L114 > > This code sleeps 100ms if no data is available for reading on a socket > (and erroneously logs "nothing to read for 100 seconds" after 10,000 100ms > loops when really about 1000 seconds will have passed - but that's just a > logging error and unlikely to be hit). This is a pretty strange and > suboptimal way to do socket programming as they're basically using > non-blocking sockets (SocketChannels) as blocking sockets. There are lots > of similar oddities strewn throughout the code base. > > > Sorry I don't have time to dig into the performance regression - just > wanted to give you a heads up about your client of choice. > > On Wednesday, June 19, 2013 1:08:31 PM UTC-7, [email protected] wrote: >> >> Here is some more information regarding the performance degradation I see >> on beanstalk 1.9. >> >> Looking at the code, I suspect the culprit is this change >> https://github.com/kr/beanstalkd/commit/1d191ba26b20f402cc8e8ee3a7f7b0de9f1ff78c >> >> Here are my numbers: >> >> Task 1.8 1.9 enqueue 600 items each in 3000 tubes 58.2 128.8 enqueue >> 500,000 items in 1 tube, then dequeue 21.15 46.15 enqueue 500,000 items >> in 1 tube, then dequeue (after enqueue 600 items each in 3000 tubes) >> 79.03 88.2 enqueue 600 items each in 3000 tubes then dequeue 152.13 >> 313.55 All timings are in seconds. Tests were run 3 times and >> averaged. Raw numbers were consistent. >> Tests were to a local beanstalkd on a 2013 macbook pro 15. >> Separate connections for each tube. All connections left open for the >> duration in all cases. Work performed using 20 thread threadpools in >> java Job is: "This is a job" >> >> Here is a gist of the java test class. It requires the Trendrr >> beanstalk client and commons-logging. >> https://gist.github.com/mattcross/5817503 >> >> If someone could comment on whether this problem is on anyone's radar I'd >> really appreciate it. >> >> -matt >> >> P.S. I also see maxed out CPU for a much higher percentage of the time >> on 1.9 but I have not characterized it. In some cases the CPU stays high >> permanently after all queues have emptied but clients are still connected. >> > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/beanstalk-talk. For more options, visit https://groups.google.com/groups/opt_out.
