Wow, lots of new information... I will briefly reply, attempting to inline my replies into context, and will share some new discovery at the end, so bear with me :)
> > As François mentioned, using #select_map would reduce the number of > intermediate objects created (if sequel_pg is used), but can't > integrate with prepared statements You nailed it. #select_map method itself is nice, but it does not work with PreparedStatements (I was silly enough to try that before reading your reply). > > It's not surprising that ActiveRecord is faster in that case, since > you are just using the connection and not ActiveRecord itself. With > Sequel, that's basically the same thing as: Um... sorry, that is just a misunderstanding, my old ActiveRecord code was really-truly ActiveRecord code, this snippet is part of my new 'attempts' at optimizing, it is simply stealing database connection from ActiveRecord, and I am aware of it :) > This looks like memprof output for the old model query instead of the > new dataset query. So for further tuning you are going to want to > submit the memprof output for the newer query. That is because this memprof dump is for whole action, and there are 8 queries there - 3 are using models, and 5 are simple id-fetchers that do not use models Anyway - here is a punchline: I finally broke loose from my 5-years old rails app and created new sinatra app to test this action (and to start core of new service api). This new sinatra app is using all the mentioned Sequel code to access same PostgreSQL database, and is running on ruby 1.9.3 (versus 1.8.7 before). As soon as I refactored code to work in new app, I clocked it at 80-90 hits/second (double of previous throughput), while total process memory consumption dropped from 200MB to 50MB. Further profiling showed that in rubyprof cost of single action dropped from 40k to 10k, and GC analysis showed that total time spent doing GC for 500 requests dropped from 2-3 seconds to 300ms. Those new numbers are good for me as they are, so I guess I am going to stop trying to squeeze any more performance out of this action, and will build rest of the app in a similar way (sequel + prepared statements). So... if anybody dealing with similar issues will find this thread, do yourself a favor, dump ruby 1.8 and jump to 1.9.3 - there is real benefit of doing so. I also gave a quick try to jruby 1.6.5, but for some reason it gave me even worse results with same gemset (25 hits/second after significant warmup), and when I tried to use jdbc driver, app would consistently crash after slowly processing about 4 requests (avg times in 200ms ballpark, message "error for object 0x10200fb10: pointer being freed was not allocated"), so I abandoned whole attempt to use JRuby. It is likely I was doing something wrong with it, but since 1.9.3 worked out so well, I was not REALLY motivated to sort those crashes out. It is a bummer that memprof is not working with ruby 1.9, I started to like having that information available. Anyway, huge thanks to everybody who chimed in here, especially Jeremy! Sequel has proved that it is a great choice for DB access layer, and it will be utilized in those new projects that we are starting now. Best regards, Olek -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sequel-talk?hl=en.
