PERFECT! Thanks so much. Worked like a charm. As an aside, how are folks setting up their tests for sphinx -- a "real" sphinx index dataset to test against? A set of fixtures? Factory-generated data?
On Jun 26, 10:44 pm, Pat Allan <[email protected]> wrote: > Hi John Paul > > I'm guessing you're using fixtures in your test database? The way > Rails assigns ID values doesn't work well with the default Sphinx > approach. You'll need to add a line or two to your config/sphinx.yml > file (or create it if you don't have one): > > test: > sql_range_step: 1000000 > > Sphinx normally grabs a thousand records at a time to process, but > with fixtures, you may have a record with id 10, and then the *next* > record is 1,000,000 - and so Sphinx is doing a lot of unneccessary > requests checking for all the ids inbetween. Setting the step size to > something much larger gets around that issue. > > Hope this helps. > > Cheers > > -- > Pat > > On 26/06/2009, at 1:15 PM, John Paul wrote: > > > > > > > Folks, > > > TS is working great in my development environment, but indexing a > > *tiny* test database takes several orders of magnitude more time. > > Here's the development rake ts:index > > > using config file /trunk/config/development.sphinx.conf'... > > indexing index 'project_core'... > > collected 217 docs, 0.6 MB > > collected 0 attr values > > sorted 0.0 Mvalues, 100.0% done > > sorted 2.2 Mhits, 98.4% done > > total 217 docs, 622462 bytes > > total 2.264 sec, 274918.75 bytes/sec, 95.84 docs/sec > > > indexing index 'profile_core'... > > collected 897 docs, 0.1 MB > > collected 0 attr values > > sorted 0.0 Mvalues, 100.0% done > > sorted 0.2 Mhits, 99.1% done > > total 897 docs, 70051 bytes > > total 1.260 sec, 55592.02 bytes/sec, 711.85 docs/sec > > > and here's rake ts:index RAILS_ENV=test > > > indexing index 'project_core'... > > collected 4 docs, 0.0 MB > > collected 0 attr values > > sorted 0.0 Mvalues, 100.0% done > > sorted 0.0 Mhits, 99.1% done > > total 4 docs, 100 bytes > > total 736.249 sec, 0.14 bytes/sec, 0.01 docs/sec > > > indexing index 'profile_core'... > > collected 6 docs, 0.0 MB > > collected 0 attr values > > sorted 0.0 Mvalues, 100.0% done > > sorted 0.0 Mhits, 98.6% done > > total 6 docs, 212 bytes > > total 3026.660 sec, 0.07 bytes/sec, 0.00 docs/sec > > > So 2s to index 217 projects and 736s to index 4... Any ideas? > > > Config files are autogenerated by rake (and I've wiped them out and > > regenned a few times) > > > Any thoughts? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
