s/wrapper/monitor_script/g
On 11/2/11, goran kent <[email protected]> wrote: > Thanks to all for responding. > > OK, to summarise (and in the interests of keeping things simple): > > merging/updating indexers: > ------------------------------------- > > touch $idx_lockfile > rm -rf $index > mv $fresh $index > unlink $idx_lockfile > touch $idx_has_changed > # ie, don't bother with symlinks and whatnot since active searchers > have cached everything. > > searchers: > -------------- > > loop > $lockfound = 0; > while (-f $idx_lockfile) { > # pause for 100ms > select(undef, undef, undef, 0.1); } > $lockfound = 1; > } > > if ($lockfound) { > $lockfound = 0; > # restart_searcher > goto restart; > } > end > > As I'm typing this though, I'm thinking, what about > Lucy::Search::IndexSearcher->serve? > It's in a loop hiding it's internals - how do I get *it* to honour my > $idx_lockfile? > > Hack LucyX/Remote/SearchServer.pm's sub serve()? Perhaps between > > read( $client_sock, $buf, $len ); > + sleep while $idx_lockfile is extant > my $response = $dispatch{$method}->( $self, thaw($buf) ); > > My final question: Peter says "For full index swap-outs, I just let > existing Searchers finish and re-open > themselves when they realize the underlying index is different" -- > peter are you saying LucyX::Remote::SearchServer does this > automatically on it's own without intervention needed, or should I > have a wrapper with LucyX::Remote::SearchServer ensconced which > monitors the indexes and kills/restarts LucyX::Remote::SearchServer > when a changed index is detected (-f $idx_has_changed)? > > urgh, this is getting dirty ugly. > > /throws toys out cot, sobs > > On 11/2/11, Peter Karman <[email protected]> wrote: >> Marvin Humphrey wrote on 11/1/11 12:36 PM: >>> On Tue, Nov 01, 2011 at 11:37:30AM +0200, goran kent wrote: >>>> I'm pondering about the best approach to handle fresh/stale indexes >>>> while searches are happening (ie, we don't want to disrupt an active >>>> search). >>> >>> Once an IndexSearcher has opened successfully, it no longer needs access >>> to >>> the index dir[1] -- you can actually wipe the index dir and the >>> IndexSearcher >>> will keep functioning indefinitely. This is achieved by caching open >>> filehandles to all files the IndexSearcher could ever need access to, so >>> that >>> it will still have access even if the files are deleted[2]. >>> >> >> fwiw, this is exactly what I do. >> >> For incremental updates, my Searcher wrapper code[1] detects when the >> index >> has >> changed and re-opens itself. >> >> For full index swap-outs, I just let existing Searchers finish and >> re-open >> themselves when they realize the underlying index is different. When the >> open >> filehandles eventually close, then the filesystem actually removes the >> old >> index. >> >> [1] https://metacpan.org/module/SWISH::Prog::Lucy::Searcher >> >> -- >> Peter Karman . http://peknet.com/ . [email protected] >> >
