Thanks for the fix Brian, I'd completely forgotten about how slow
#attributes is. I've merged in the changes and pushed them. For people
waiting for the updated gem, I'm hoping to look at other changes
today, but even if I don't, goal is to update version number within 24
hours.
--
Pat
On 15/08/2009, at 1:02 AM, Brian Terlson wrote:
>
> Since updating Thinking Sphinx to the latest version, my app has seen
> some very marked slowdowns when dealing with thousands of results.
> Where I used to be able to get results in a few seconds, response
> times are now well into the multiple minute range. After some poking
> around and benchmarking, I've narrowed down the problem to
> #primary_key_for_sphinx. This method is called inside loops in
> Search's instances_from_matches and instances_from_class methods, the
> latter loop being O(n^2).
>
> I've benchmarked the difference between #primary_key_for_sphinx and
> calling .id straight up, as used to be done prior to commit 9191a5d5,
> and primary_key_for_sphinx is over 400 times slower.
>
>>> Benchmark.bmbm do |b|
> ?> b.report("Primary key for sphinx") { 10000.times
> { c.primary_key_for_sphinx } }
>>> b.report("Primary key") { 10000.times { c.id } }
>>> end
>
> Rehearsal ----------------------------------------------------------
> Primary key for sphinx 4.080000 0.370000 4.450000 ( 4.547231)
> Primary key 0.010000 0.000000 0.010000 ( 0.020685)
> ------------------------------------------------- total: 4.460000sec
>
> user system total real
> Primary key for sphinx 3.880000 0.370000 4.250000 ( 4.317910)
> Primary key 0.000000 0.000000 0.000000 ( 0.009915)
>
> That's 4.5 seconds for 10000 calls, which will happen in the worst
> case even with 100 records returned.
>
> The majority of that time is due to Rails' attributes method, which,
> as far as I know, constructs a new hash of all typecasted attributes
> each time it is called. The fix, therefore, seems easy enough. Since
> we only need the one attribute, not all of the attributes, we don't
> need Rails to build up our entire hash of attributes. Using
> read_attribute in place of attributes[] results in a 110x speedup in
> my tests with a pretty heavy model with lots of attributes, but even
> trim models will see a massive speed boost.
>
> I've gone ahead and implemented the change required on my github fork
> here: http://github.com/bterlson/thinking-sphinx/tree/master. Let me
> know if you have any questions!
>
> Brian Terlson
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/thinking-sphinx?hl=en
-~----------~----~----~----~------~----~------~--~---
