Did this make sense?
On Jun 15, 11:09 pm, Alex <[email protected]> wrote:
> This is what I am doing (trying anyway) in production:
>
> 1) Full index every 24 hours
> 2) Delta index every 20 minutes with threshold set at 30 minutes
> (after an initial full index)
>
> I end up with the most recent full index (up to 24 hours old) plus the
> most recent delta index regardless of previous delta indexes. So
> after 2 hours, I have everything up to 2 hours old, plus everything 30
> minutes old, but nothing in between.
>
> I tested this by:
>
> 1) Running a full index
> 2) Waiting a few hours
> 3) Running a delta index with a threshold of 1 hour
> 4) Searching for a record 45 minutes old and finding it
> 5) Without doing anything else except reconfiguring, running a delta
> index with a threshold of 1 minute
> 6) Searching for a record 45 minutes old and NOT finding it
>
> This indicates to me that the delta index is not being merged in.
>
> I'm also wondering how the index handles duplicates under this
> scenario since it will index records multiple times and there doesn't
> seem to be any id based de-duping code.
>
> On Jun 15, 7:34 pm, Pat Allan <[email protected]> wrote:
>
> > I don't think I'm quite understanding the situation, so forgive me if
> > I sound a little condescending...
>
> > Are you saying you've not run a main index for over ten hours, and
> > then run a delta index (with threshold set to 1 hour), and you don't
> > get the changes that have happened since the main index but before the
> > last hour? That's the expected outcome - the delta index task needs to
> > be run at the same regularity as the threshold.
>
> > Or have I got it completely wrong?
>
> > --
> > Pat
>
> > On 15/06/2009, at 10:24 PM, Alex wrote:
>
> > > Possibly. But basically if I have records spanning the last 10 hours,
> > > and I set the threshold at 12 hours, they all get indexed with the
> > > main index. If I change the threshold to 1 hour, then only the last
> > > hour plus the main index shows up.
>
> > > -Alex
>
> > > On Jun 15, 6:28 pm, Pat Allan <[email protected]> wrote:
> > >> Hi Alex
>
> > >> The output looks right... is it possible for you to wrap up the
> > >> repeatable test in some way so I can run it on my machine?
>
> > >> --
> > >> Pat
>
> > >> On 15/06/2009, at 8:45 PM, Alex wrote:
>
> > >>> Here is the output from rake ts:in:delta :
>
> > >>> (in /mnt/app/releases/20090610140740)
> > >>> Sphinx 0.9.8.1-release (r1533)
> > >>> Copyright (c) 2001-2008, Andrew Aksyonoff
>
> > >>> using config file '/mnt/app/releases/20090610140740/config/
> > >>> production.sphinx.conf'...
> > >>> indexing index 'source_listing_delta'...
> > >>> collected 41730 docs, 180.2 MB
> > >>> collected 0 attr values
> > >>> sorted 0.0 Mvalues, 100.0% done
> > >>> sorted 26.6 Mhits, 100.0% done
> > >>> total 41730 docs, 180158829 bytes
> > >>> total 123.160 sec, 1462798.97 bytes/sec, 338.83 docs/sec
> > >>> rotating indices: succesfully sent SIGHUP to searchd (pid=5673).
> > >>> Sphinx 0.9.8.1-release (r1533)
> > >>> Copyright (c) 2001-2008, Andrew Aksyonoff
>
> > >>> using config file '/mnt/app/releases/20090610140740/config/
> > >>> production.sphinx.conf'...
> > >>> merged 3294.0 Kwords
> > >>> merged in 167.3 sec
> > >>> rotating indices: succesfully sent SIGHUP to searchd (pid=5673).
>
> > >>> On Jun 15, 4:59 pm, Alex <[email protected]> wrote:
> > >>>> I'm using sphinx-0.9.8.1 and it is definitely not working - I have
> > >>>> done a number of repeatable tests.
>
> > >>>> Sorry for the long response time - I've been traveling.
>
> > >>>> On Jun 10, 5:16 pm, Pat Allan <[email protected]> wrote:
>
> > >>>>> Hi Alex
>
> > >>>>> Yes, it should be merging, so something's not quite right. What
> > >>>>> version of Sphinx are you running?
>
> > >>>>> --
> > >>>>> Pat
>
> > >>>>> On 10/06/2009, at 9:52 AM, Alex wrote:
>
> > >>>>>> When I set a a threshold of 10.minutes, I get the last full re-
> > >>>>>> index
> > >>>>>> plus the last 10 minutes of updates. Re-running rake:in:delta
> > >>>>>> replaces the previous delta instead of adding to it. That
> > >>>>>> means my
> > >>>>>> threshold needs to be set according to full reindexing periods,
> > >>>>>> not
> > >>>>>> delta periods, which throws everything off.
>
> > >>>>>> Isn't this supposed to merge the delta into the main index? That
> > >>>>>> doesn't seem to be working as expected.
>
> > >>>>>> -Alex
>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/thinking-sphinx?hl=en
-~----------~----~----~----~------~----~------~--~---
