While I don't doubt there's been a slowdown, and your benchmarks are useful, the Trendrr client has some glaring bugs and strange code that make it less than an ideal test client. For example when reading a response from beanstalk it has the following strange loop:
https://github.com/dustismo/TrendrrBeanstalk/blob/master/src/com/trendrr/beanstalk/BeanstalkConnection.java#L101-L114 This code sleeps 100ms if no data is available for reading on a socket (and erroneously logs "nothing to read for 100 seconds" after 10,000 100ms loops when really about 1000 seconds will have passed - but that's just a logging error and unlikely to be hit). This is a pretty strange and suboptimal way to do socket programming as they're basically using non-blocking sockets (SocketChannels) as blocking sockets. There are lots of similar oddities strewn throughout the code base. Sorry I don't have time to dig into the performance regression - just wanted to give you a heads up about your client of choice. On Wednesday, June 19, 2013 1:08:31 PM UTC-7, [email protected] wrote: > > Here is some more information regarding the performance degradation I see > on beanstalk 1.9. > > Looking at the code, I suspect the culprit is this change > https://github.com/kr/beanstalkd/commit/1d191ba26b20f402cc8e8ee3a7f7b0de9f1ff78c > > Here are my numbers: > > Task 1.8 1.9 enqueue 600 items each in 3000 tubes 58.2 128.8 enqueue > 500,000 items in 1 tube, then dequeue 21.15 46.15 enqueue 500,000 items > in 1 tube, then dequeue (after enqueue 600 items each in 3000 tubes) 79.03 > 88.2 enqueue 600 items each in 3000 tubes then dequeue 152.13 313.55 All > timings are in seconds. Tests were run 3 times and averaged. Raw numbers > were consistent. > Tests were to a local beanstalkd on a 2013 macbook pro 15. > Separate connections for each tube. All connections left open for the > duration in all cases. Work performed using 20 thread threadpools in > java Job is: "This is a job" > > Here is a gist of the java test class. It requires the Trendrr > beanstalk client and commons-logging. > https://gist.github.com/mattcross/5817503 > > If someone could comment on whether this problem is on anyone's radar I'd > really appreciate it. > > -matt > > P.S. I also see maxed out CPU for a much higher percentage of the time on > 1.9 but I have not characterized it. In some cases the CPU stays high > permanently after all queues have emptied but clients are still connected. > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/beanstalk-talk. For more options, visit https://groups.google.com/groups/opt_out.
