Hi Adam The kill list conf option is in TS (was added a couple of months ago, I think). However, if you wanted a default option for deltas, then it really only applies to the datetime deltas (in other deltas, you don't get the doubling up, and there's no easy way to figure out which documents to delete either).
So: a patch is definitely welcome, and you'll want to add it to ts-datetime-delta, not the thinking-sphinx gem. Great detective work by both you and Steven. Cheers -- Pat On 27/03/2010, at 12:15 PM, adamcooper wrote: > We've been looking at this issue fairly in depth and it seems to be > due to a datetime-delta implementation issue (not sure if it's only > specific to ours...) > > The real crux of the issue is because the main and delta index overlap > and therefore double the facet counts. > > After a full index, the delta and main index have an overlap of > records that have been updated in last hour (or threshold time). So > any facet counting on those records get doubled. Normal searches are > not affected. Counts are further messed up as records that were > originally only in the main index are updated and appear in the delta > index and therefore become double counted. I can explain this issue > further if it's not totally clear. Index merging further causes > issues. > > We have manually tweaked the conf file to get the correct behaviour > using sql_query_killlist and sql_query_post and an intermediate > table. Basically, we track the last full index time and add any > records that have been updated since then to the delta killlist. > We'll probably add the deleted records to there too (based on an > acts_as_paranoid type deletion). > > Does anyone know if there is an easier way to accomplish this? Or is > this what others are doing too? > > Pat, we are looking to patch Thinking Sphinx to add killlist support, > have you done any initial thinking as to how you would want it added > or how it best could be added? > > Thanks, > Adam > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
