Since updating Thinking Sphinx to the latest version, my app has seen
some very marked slowdowns when dealing with thousands of results.
Where I used to be able to get results in a few seconds, response
times are now well into the multiple minute range. After some poking
around and benchmarking, I've narrowed down the problem to
#primary_key_for_sphinx. This method is called inside loops in
Search's instances_from_matches and instances_from_class methods, the
latter loop being O(n^2).
I've benchmarked the difference between #primary_key_for_sphinx and
calling .id straight up, as used to be done prior to commit 9191a5d5,
and primary_key_for_sphinx is over 400 times slower.
>> Benchmark.bmbm do |b|
?> b.report("Primary key for sphinx") { 10000.times
{ c.primary_key_for_sphinx } }
>> b.report("Primary key") { 10000.times { c.id } }
>> end
Rehearsal ----------------------------------------------------------
Primary key for sphinx 4.080000 0.370000 4.450000 ( 4.547231)
Primary key 0.010000 0.000000 0.010000 ( 0.020685)
------------------------------------------------- total: 4.460000sec
user system total real
Primary key for sphinx 3.880000 0.370000 4.250000 ( 4.317910)
Primary key 0.000000 0.000000 0.000000 ( 0.009915)
That's 4.5 seconds for 10000 calls, which will happen in the worst
case even with 100 records returned.
The majority of that time is due to Rails' attributes method, which,
as far as I know, constructs a new hash of all typecasted attributes
each time it is called. The fix, therefore, seems easy enough. Since
we only need the one attribute, not all of the attributes, we don't
need Rails to build up our entire hash of attributes. Using
read_attribute in place of attributes[] results in a 110x speedup in
my tests with a pretty heavy model with lots of attributes, but even
trim models will see a massive speed boost.
I've gone ahead and implemented the change required on my github fork
here: http://github.com/bterlson/thinking-sphinx/tree/master. Let me
know if you have any questions!
Brian Terlson
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/thinking-sphinx?hl=en
-~----------~----~----~----~------~----~------~--~---
