Hello First of all, let me express my gratitude for Sequel. I spent last week or so working closely with it for performance-critical app, and I have to say that it is a pleasure to work with Sequel.
Those words are coming from somebody that used Rails and ActiveRecord for the last 4-5 years, starting from 1.x and all the way up to 3.0. I think that ActiveRecord is moving along in the right direction, splitting itself into low level SQL operations (AREL) and high level mapping, much like Sequel. Difference is that, at the very least, in current version (3.0) of ActiveRecord, it is very inconsistent and it drives me nuts trying to use it, and Sequel is ... ActiveRecord 3.0 done right :) Somehow whenever I need something from Sequel, it is there, it is easy to find and it works like expected, with the bare minimum of magic. Very refreshing feeling. So anyway, I used Sequel models and bound variables, and prepared statements, and found that while my 'best case scenario' action response times went down about 2-3 times, and profiling with ruby-prof proves about same kind of improvement, overall response time under stress DROPPED about 20%. After much of investigation it all boiled down for the fact that for one reason or another Sequel is more wasteful with memory, comparing to my old ActiveRecord code. Essentially, my new code is generating DOUBLE of wasted memory that needs to be garbage collected, and my garbage collection times are starting to become large then my 'useful computing' times. For 500 executions of my old code about 350MB of memory would be 'spent and collected', and for new code that number went up to 600MB, number of allocations themselves went from 7M to 9M. That problem could be overcome to some extent by using patched version of ruby that allows fine tuning of garbage collection, and I tested that by switching from MRI 1.8.7 to REE 1.8.7. Yes, definite improvements, and at the end my app is faster than it ever was... but still good chunk of time is wasted in GC, and this kind of 'solution' is not quite right. So, to sum it up, it appears that somewhere inside Sequel code is generating a bunch of short-lived objects, more then it really should (based on GC stats that I have seen, Strings and Arrays are most likely culprit). It is not the code that generates queries, for sure, since I am using prepared statements 100%, it is code that is related to loading data. BTW - on a related note - Sequel::Timezones#convert_timestamp is one of the most expensive functions there, it takes about 0.3ms to run it just once, so for table containing 6 DateTime columns, total time spent adjusting timezones is about 2ms, which is not so good considering that the time to run query against database was about 0.6ms. I understand that maybe what I am doing is a bit atypical to usual ruby web projects, my action response time in question is 12ms now (GC excluded), not that many people care to optimize it that low, while still sticking with SQL databases (most would switch away to no-SQL like Mongo, Redis etc). Still, I think that if we could reduce memory usage of Sequel a bit over time, everybody would benefit. I know it is not simple effort and do not expect quick and magical cure, but based on comparison with ActiveRecord, this is one of the few areas where Sequel could improve a bit. Just wanted to share my findings with Sequel's maintainer and community. Keep up the good work, guys! If I am sorry about anything now, that would be "why, oh why, I have not discovered Sequel 3 years ago???" :) -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sequel-talk?hl=en.
