Thanks Pat, In regards to speed, is there anything I can do, or is it just a limitation of sphinx? Also, any idea why the db is getting hit so hard. I haven't got a chance to look into the logs yet to see if its running any slows queries, but I would imagine its just reads. Whenever the indexer is running, I see a lot of DB timeouts from other services, so I can't help but feel that something is up.
Also, in regards to changing the structure of my indexes, so I need to run rebuild, or can I just configure and restart (i.e. must I reindex before restarting with a new config file)? Thanks. On Feb 17, 3:21 am, "Pat Allan" <[email protected]> wrote: > It's best to use the ts:index task in cron, because all that does is reindex > the data. ts:rebuild stops Sphinx, reindexes, and *then* restarts Sphinx - > and you really don't need to restart Sphinx except when the structure of your > indices change (you've added an index to another model, or added a new field > or attribute to an existing model, a new config setting, etc). > > So, I'd recommend switching to ts:index - although it won't be much different > in speed (starting/stopping Sphinx won't take much time), but it'll mean > people can still search while the indexing is happening. > > -- > Pat > > On 17/02/2012, at 7:52 PM, gmoniey wrote: > > > > > > > > > For the past few years, I have had a cron job that wakes up around > > 4am, and runs the ts:rebuild rake task. Unfortunately, the size of all > > of my indexes has grown significantly, and this job takes about 40-60 > > minutes, and the load on the database is pretty significant. I'm > > indexing roughly 2-3 million records (some of which have pretty > > detailed indexes defined). > > > I've read on some blogs the index/reindex is significantly faster than > > rebuild, but haven't been able to find any formal documentation > > mentioning that. > > > Do I have any other options? > > > Thanks. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Thinking Sphinx" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/thinking-sphinx?hl=en. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
