Hi Sundar Just pushed a tweak to ts-delayed-delta to only flag records as deleted in Sphinx if they’re still marked as in the delta. This should help your situation, I think?
It’s discussed here on GitHub: https://github.com/pat/ts-delayed-delta/issues/49 <https://github.com/pat/ts-delayed-delta/issues/49> Cheers — Pat > On 3 Jul 2015, at 7:58 am, Sundar Rajamanickam > <[email protected]> wrote: > > Pat > > Thanks for the reply. What if we flag the documents in the core as deleted > after the delta indexing? > > And also I came across one more scenario: > > During full indexer run, if an object gets updated - DeltaJob and > FlagAsDeletedJob get enqueued. > > Full indexer resets the 'delta' attribute of the object to false - that the > DeltaJob ignores the updated object and FlagAsDeletedJob flags the > corresponding document as deleted in the core index. Now the document remains > unavailable in both core and delta indices till the next full indexing. > > Please note that we have a single queue in the DJ for processing sphinx jobs. > > Thanks > Sundar Rajamanickam > > On Monday, June 29, 2015 at 5:01:13 PM UTC-7, Sundar Rajamanickam wrote: > Hi > > I have a hourly (cron based) full indexer and delayed delta indexer setup. > > Say, when I update an object of 'Article' class - two delayed jobs get > enqueued - DeltaIndexJob and FlagAsDeletedJob. > > If only the FlagAsDeletedJob gets executed, and DeltaIndexJob does not get > executed. Now when the full indexer starts, it cleans up all the existing > jobs in the DJ queue - means the updated 'Article' object will not be > available in both the core and delta indices till the full indexing completes. > > Our full indexer runs for about 12-15 minutes everytime because of the size > of the database. This results in a significant delay, that the object becomes > available for search. > > Ensuring that the FlagAsDeletedJob runs after the DeltaIndexJob will help > reduce the impact, right? > > Thanks > Sundar Rajamanickam > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at http://groups.google.com/group/thinking-sphinx > <http://groups.google.com/group/thinking-sphinx>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/thinking-sphinx. For more options, visit https://groups.google.com/d/optout.
