Hi Alex Yup, it makes sense, I just need to find some time to look into it. I've got a ton of client work at the moment though, so I'm afraid it's probably not going to happen until sometime in July. Sorry.
-- Pat On 18/06/2009, at 1:13 PM, Alex wrote: > > Did this make sense? > > On Jun 15, 11:09 pm, Alex <[email protected]> wrote: >> This is what I am doing (trying anyway) in production: >> >> 1) Full index every 24 hours >> 2) Delta index every 20 minutes with threshold set at 30 minutes >> (after an initial full index) >> >> I end up with the most recent full index (up to 24 hours old) plus >> the >> most recent delta index regardless of previous delta indexes. So >> after 2 hours, I have everything up to 2 hours old, plus everything >> 30 >> minutes old, but nothing in between. >> >> I tested this by: >> >> 1) Running a full index >> 2) Waiting a few hours >> 3) Running a delta index with a threshold of 1 hour >> 4) Searching for a record 45 minutes old and finding it >> 5) Without doing anything else except reconfiguring, running a delta >> index with a threshold of 1 minute >> 6) Searching for a record 45 minutes old and NOT finding it >> >> This indicates to me that the delta index is not being merged in. >> >> I'm also wondering how the index handles duplicates under this >> scenario since it will index records multiple times and there doesn't >> seem to be any id based de-duping code. >> >> On Jun 15, 7:34 pm, Pat Allan <[email protected]> wrote: >> >>> I don't think I'm quite understanding the situation, so forgive me >>> if >>> I sound a little condescending... >> >>> Are you saying you've not run a main index for over ten hours, and >>> then run a delta index (with threshold set to 1 hour), and you don't >>> get the changes that have happened since the main index but before >>> the >>> last hour? That's the expected outcome - the delta index task >>> needs to >>> be run at the same regularity as the threshold. >> >>> Or have I got it completely wrong? >> >>> -- >>> Pat >> >>> On 15/06/2009, at 10:24 PM, Alex wrote: >> >>>> Possibly. But basically if I have records spanning the last 10 >>>> hours, >>>> and I set the threshold at 12 hours, they all get indexed with the >>>> main index. If I change the threshold to 1 hour, then only the >>>> last >>>> hour plus the main index shows up. >> >>>> -Alex >> >>>> On Jun 15, 6:28 pm, Pat Allan <[email protected]> wrote: >>>>> Hi Alex >> >>>>> The output looks right... is it possible for you to wrap up the >>>>> repeatable test in some way so I can run it on my machine? >> >>>>> -- >>>>> Pat >> >>>>> On 15/06/2009, at 8:45 PM, Alex wrote: >> >>>>>> Here is the output from rake ts:in:delta : >> >>>>>> (in /mnt/app/releases/20090610140740) >>>>>> Sphinx 0.9.8.1-release (r1533) >>>>>> Copyright (c) 2001-2008, Andrew Aksyonoff >> >>>>>> using config file '/mnt/app/releases/20090610140740/config/ >>>>>> production.sphinx.conf'... >>>>>> indexing index 'source_listing_delta'... >>>>>> collected 41730 docs, 180.2 MB >>>>>> collected 0 attr values >>>>>> sorted 0.0 Mvalues, 100.0% done >>>>>> sorted 26.6 Mhits, 100.0% done >>>>>> total 41730 docs, 180158829 bytes >>>>>> total 123.160 sec, 1462798.97 bytes/sec, 338.83 docs/sec >>>>>> rotating indices: succesfully sent SIGHUP to searchd (pid=5673). >>>>>> Sphinx 0.9.8.1-release (r1533) >>>>>> Copyright (c) 2001-2008, Andrew Aksyonoff >> >>>>>> using config file '/mnt/app/releases/20090610140740/config/ >>>>>> production.sphinx.conf'... >>>>>> merged 3294.0 Kwords >>>>>> merged in 167.3 sec >>>>>> rotating indices: succesfully sent SIGHUP to searchd (pid=5673). >> >>>>>> On Jun 15, 4:59 pm, Alex <[email protected]> wrote: >>>>>>> I'm using sphinx-0.9.8.1 and it is definitely not working - I >>>>>>> have >>>>>>> done a number of repeatable tests. >> >>>>>>> Sorry for the long response time - I've been traveling. >> >>>>>>> On Jun 10, 5:16 pm, Pat Allan <[email protected]> wrote: >> >>>>>>>> Hi Alex >> >>>>>>>> Yes, it should be merging, so something's not quite right. What >>>>>>>> version of Sphinx are you running? >> >>>>>>>> -- >>>>>>>> Pat >> >>>>>>>> On 10/06/2009, at 9:52 AM, Alex wrote: >> >>>>>>>>> When I set a a threshold of 10.minutes, I get the last full >>>>>>>>> re- >>>>>>>>> index >>>>>>>>> plus the last 10 minutes of updates. Re-running rake:in:delta >>>>>>>>> replaces the previous delta instead of adding to it. That >>>>>>>>> means my >>>>>>>>> threshold needs to be set according to full reindexing >>>>>>>>> periods, >>>>>>>>> not >>>>>>>>> delta periods, which throws everything off. >> >>>>>>>>> Isn't this supposed to merge the delta into the main index? >>>>>>>>> That >>>>>>>>> doesn't seem to be working as expected. >> >>>>>>>>> -Alex >> >> > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
