Hi John Paul I'm guessing you're using fixtures in your test database? The way Rails assigns ID values doesn't work well with the default Sphinx approach. You'll need to add a line or two to your config/sphinx.yml file (or create it if you don't have one):
test: sql_range_step: 1000000 Sphinx normally grabs a thousand records at a time to process, but with fixtures, you may have a record with id 10, and then the *next* record is 1,000,000 - and so Sphinx is doing a lot of unneccessary requests checking for all the ids inbetween. Setting the step size to something much larger gets around that issue. Hope this helps. Cheers -- Pat On 26/06/2009, at 1:15 PM, John Paul wrote: > > Folks, > > TS is working great in my development environment, but indexing a > *tiny* test database takes several orders of magnitude more time. > Here's the development rake ts:index > > using config file /trunk/config/development.sphinx.conf'... > indexing index 'project_core'... > collected 217 docs, 0.6 MB > collected 0 attr values > sorted 0.0 Mvalues, 100.0% done > sorted 2.2 Mhits, 98.4% done > total 217 docs, 622462 bytes > total 2.264 sec, 274918.75 bytes/sec, 95.84 docs/sec > > indexing index 'profile_core'... > collected 897 docs, 0.1 MB > collected 0 attr values > sorted 0.0 Mvalues, 100.0% done > sorted 0.2 Mhits, 99.1% done > total 897 docs, 70051 bytes > total 1.260 sec, 55592.02 bytes/sec, 711.85 docs/sec > > and here's rake ts:index RAILS_ENV=test > > indexing index 'project_core'... > collected 4 docs, 0.0 MB > collected 0 attr values > sorted 0.0 Mvalues, 100.0% done > sorted 0.0 Mhits, 99.1% done > total 4 docs, 100 bytes > total 736.249 sec, 0.14 bytes/sec, 0.01 docs/sec > > indexing index 'profile_core'... > collected 6 docs, 0.0 MB > collected 0 attr values > sorted 0.0 Mvalues, 100.0% done > sorted 0.0 Mhits, 98.6% done > total 6 docs, 212 bytes > total 3026.660 sec, 0.07 bytes/sec, 0.00 docs/sec > > So 2s to index 217 projects and 736s to index 4... Any ideas? > > Config files are autogenerated by rake (and I've wiped them out and > regenned a few times) > > Any thoughts? > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
