Hi Pat, Good point, for most use-cases I agree (and will delta indexing be fast). In my case I have a 20GB database which is updated from an external source with thousands of updates throughout the day. Currently these updates are only reindexed in the night, however some new data during the day would be nice. With suspended delta I could keep using the main sphinx index and merge delta e.g. every hour, however merging indexes is also quite expensive as the main sphinx index is many GB in size. That's how I came up with the suggested approach of using delayed_delta but not hitting expensive delta updates too often/continuously (delta indexes can after thousands of updates during the day also take say 15 minutes). Given the above you maybe have a better suggestion/approach for me? Otherwise I do agree it might be a good idea to be able to set such a delay at least per model. Very much interested in your opinion on this! Thanks,
Gyuri On Mar 22, 9:27 pm, Pat Allan <[email protected]> wrote: > Hi Gyuri > > An interesting idea. My one concern is that people expect search updates to > be as fast as possible - so if we're pushing it out to 30 minutes, that's > less than ideal. > > Would it be better to only delay the job by the given window (I'd default it > to a few minutes) if there's an existing job in the queue? That way, you > shouldn't end up with any more than two items in the queue, right? > > -- > Pat > > On 20/03/2011, at 3:52 PM, vdbilt wrote: > > > > > > > > > Hi Pat, > > > Having looked at suspended_delta to prevent continuous delta indexing > > during bulk updates, I came up with a possibly useful approach when > > you're using delayed_delta. This doesn't define a block that runs > > before indexing occurs, but it makes sure indexing is only run every X > > minutes at max. Interested to hear your opinion / possible objections > > on this approach (changing the behaviour of ts_delayed_delta): > >https://gist.github.com/878738 > > > Thanks, > > Gyuri > > > On Feb 9, 1:26 pm, Pat Allan <[email protected]> wrote: > >> Hi Calvin > > >> That's a smart idea with using the delta flags as a reliable indicator of > >> what to remove from the core index. Hadn't thought of that. Can you log an > >> issue on Github for it? You're welcome to fork and patch too, if you'd > >> like :) > > >> Cheers > > >> -- > >> Pat > > >> On 09/02/2011, at 7:15 PM, calvin wrote: > > >>> Hi, > > >>> When using suspended_delta, any changes made within the block is not > >>> fully reflected in the index. I'm using delayed_delta. > > >>> For example: > > >>> Product.suspended_delta do > >>> ..... > >>> product.description = "some new description" > >>> product.save > >>> end > > >>> Now when you search the Product index, product is searchable by both > >>> new description and old (slate) description. This is because the > >>> record in the core index not deleted when using suspended_delta. It > >>> would seem like suspended_delta should query all objects with > >>> delta=true and call delete_in_index on each on all core indexes. > > >>> Please let me know if this suggestion make sense. > > >>> Thanks. > >>> Calvin > > >>> -- > >>> You received this message because you are subscribed to the Google Groups > >>> "Thinking Sphinx" group. > >>> To post to this group, send email to [email protected]. > >>> To unsubscribe from this group, send email to > >>> [email protected]. > >>> For more options, visit this group > >>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Thinking Sphinx" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/thinking-sphinx?hl=en. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
