Hi Gyuri Sorry for the slow response on this - long flight and jetlag plus various projects has taken its toll on TS support.
I'm not sure how much of an impact this approach would have on performance, as Sphinx handles updating multiple documents in a single call. So it really comes down to Sphinx. I think the only place you'd need to change is in ts-delayed-delta: https://github.com/freelancing-god/ts-delayed-delta/blob/master/lib/thinking_sphinx/deltas/delayed_delta.rb#L42 If there's a provided instance, then let it continue as normal. Otherwise, grab all the records that have been changed by referencing the delta column (bonus points if you can do this without actually instantiating AR objects - just grab the id column using AR::Base.connection.select_values, and then translate the values from AR's ids to sphinx document ids), and make sure the FlagAsDeletedJob accepts multiple ids. Let me know how you go :) -- Pat On 24/03/2011, at 7:47 AM, vdbilt wrote: > Hi Pat, > > That's a good idea; a lot of updates all come in at once, so waiting > until those updates finished and then updating the index would be an > even better way then using 'my' approach (because with my approach you > might start indexing while the import process is still running as the > processing duration may vary depending on the number of updates). > The only note/question I could think of: with my approach you have > thousands of single deletion jobs during the day (sounds not too > efficient ;)), with your suggested suspended_delta approach at the end > of the day you'll have tens of thousands of records with delta set > (and being part of the delta index). I have no idea if that would make > the deletion job very 'expensive'? I suspect that deleting them all at > once after reindexing will probably be much faster and more instant > (also resulting in less time there are duplicate records in both delta > and main index). > > Is this something we could easily try or would it involve quite some > changes to TS? As the resulting jobs would still take a while it's > probably a good idea to keep delayed_job to process the jobs in the > background too, right? Would that also require changes to ts-delayed- > delta? > > Thanks!!! > Gyuri > > > On Mar 23, 6:34 pm, Pat Allan <[email protected]> wrote: >> Hi Gyuri >> >> Are these updates all happening at once? Is there some modification to >> suspended_delta that will do the job? Can you update the deleted flags for >> the core index with every record that has delta set to true? This way, we >> only get two jobs (one index, one deletion), for as many updates as you like >> within the suspended_delta block. >> >> -- >> Pat > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
