Ciaran, Well, we had to try! You mentioned having some debug logging - any chance you have the log (including SQL calls) from both the AR test and the Sequel test? Maybe one or two requests?
Don On Nov 22, 2011, at 9:28 AM, Ciaran Archer wrote: > Hi Don > > That is a decent lead, so I ran the test again and I disabled the AR > query cache as you suggested: > > AR BENCHMARK RESULTS (NO QUERY CACHE) > Requests per second: 173.99 [#/sec] (mean) > Time per request: 57.476 [ms] (mean) > > So compared with the previous results: > > AR BENCHMARK RESULTS > Requests per second: 182.08 [#/sec] (mean) > Time per request: 54.922 [ms] (mean) > > SEQUEL BENCHMARK RESULTS > Requests per second: 8.72 [#/sec] (mean) > Time per request: 1146.993 [ms] (mean) > > There is no serious slowdown I'm afraid. > > My app setups are very similar, there is some different config as per > the gists I posted, but I am pretty sure that's not an issue and > should not introduce this latency. However, I might try tomorrow to > come up with a single sample app that can run both benchmarks so as to > be 100% sure it's not something I am doing in my app setup. > > But please keep the ideas coming and I will test what I can. > > Cheers, > Ciaran > > On Nov 22, 4:51 pm, Don Morrison <[email protected]> wrote: >> Ciaran, >> >> My gut reaction is that you are seeing performance gains from the >> ActiveRecord query cache. In Rails, the query cache lifecycle is scoped to >> an action but I don't know how (or if) that maps to Sinatra and the way >> you're testing. >> >> AR will cache the query results and then just return those to you rather >> than run the query again. You can verify that the query cache isn't involved >> by calling: >> >> ActiveRecord::Base.connection.disable_query_cache! >> >> in your Sinatra configuration. At least then you will have Sequel and AR on >> a level playing field since Sequel doesn't have a query cache (and my >> understanding is that it never will). I can't quite work out if this is even >> involved but better safe than sorry. >> >> All of this doesn't really speak to the problem of 8.72 requests per second. >> I assume that you have two different setups to test AR vs Sequel, and if so, >> they are exactly the same except for the ORM code? Which leads me to think >> something is running more than it should (multiple queries) or the config is >> different. >> >> I'm using Sequel in Sinatra under JRuby and couldn't be happier - I'd be >> interested in finding out what is going on, though, just to make sure I'm >> not doing the same thing. :-) >> >> Don >> >> On Nov 22, 2011, at 5:52 AM, Ciaran Archer wrote: >> >> >> >> >> >> >> >>> Hi there >> >>> We've been evaluating the use of Sequel with SQL Server, and as poart >>> of this we've been benchmarking Sequel against AR. >> >>> We're using JRuby, and Torquebox as our application server for all >>> tests. We are using Sinatra as our web framework and Apache Bench to >>> run the tests. A SQL Server 2008 R2 instance is being used for the >>> tests. >> >>> Our benchmark test is very simple, return all rows in a table and >>> print them to the standard output. The test table has 72 rows. >> >>> AR BENCHMARK RESULTS >> >>> Requests per second: 182.08 [#/sec] (mean) >>> Time per request: 54.922 [ms] (mean) >> >>> SEQUEL BENCHMARK RESULTS >> >>> Requests per second: 8.72 [#/sec] (mean) >>> Time per request: 1146.993 [ms] (mean) >> >>> I've extracted the relevant Sequel code, >>> here:https://gist.github.com/1385695 >> >>> I am trying to understand why Sequel seems to be doing so poorly and >>> I'm hoping someone can tell me where I might be introducing an issue, >>> or where I might be able to improve things. >> >>> Apologies in advance if the code is not super, I've only been using >>> Ruby a few months :) >> >>> Cheers, >>> Ciaran >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "sequel-talk" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group >>> athttp://groups.google.com/group/sequel-talk?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "sequel-talk" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sequel-talk?hl=en. > -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sequel-talk?hl=en.
