On Tue, 17 Aug 2004, Patrick Burleson wrote: > Forward back to list. > > > ---------- Forwarded message ---------- > From: Patrick Burleson <[EMAIL PROTECTED]> > Date: Tue, 17 Aug 2004 11:30:19 -0400 > Subject: Re: Swapping Indexes? > To: Stephane James Vaucher <[EMAIL PROTECTED]> > > Stephane, > > Thank you for the ideas. I'm going about implenting idea 1 (I like the > idea of leaving the temp index around for recovery), but I have a > question reguarding your original index. Do you just copy over the > temp index and don't worry abou cleaning up the old index directory?
Actually, I use a IndexWriter in overwrite mode on the master dir and merge the temp dir. This cleans up the old master. > Right now I have my code deleting the files in the main index > directory after telling the search controller to switch to the temp > index. But by doing that, I need to manage existing searches and not > break them while they are running. I also still run into the open > files problem on Windows when trying to delete a file one of the > searchers has open before it's closed. I used to way some time (~1 minute) for all searches on the old master to finish after redirecting to the temp dir, then I would switch to the new master. > Thoughts? If you apply a lease-like contract with your searchers where they borrow a reference to a searcher and then hand it back to the manager, you can probably trace your open files. HTH, sv > Patrick > > > > > On Mon, 16 Aug 2004 18:22:20 -0400 (EDT), Stephane James Vaucher > <[EMAIL PROTECTED]> wrote: > > I've tried two options that seem to work: > > > > 1) Have a singleton that is responsible that will control your searchers. > > This controller can temporarilly redirect your searchers to > > c:/temp/myindex, allowing you to copy you index to c:/myindex. After that > > process completes, your controller can tell your searchers to use > > c:/myindex, allowing you to then erase your temp index. > > > > If you index nightly, you can always *not* erase your tmp dir, your index > > process will do this automatically if you create your IndexWriter with > > the overwrite option. This way, you can have a backup index if there is > > a system failure at some point (like when you copy/move directories). > > > > 2) Use an incremental index. Regularly, I scan my files, see if there are > > modification/additions and update my master index. Removing from the > > master index, adding to a temp dir, then merging. I haven't seen any > > weirdness on windows with this process. > > > > HTH, > > sv > > > > > > > > On Mon, 16 Aug 2004, Patrick Burleson wrote: > > > > > I've read in the docs about updating an index and its suggestion > > > reguarding swapping out indexes with a directory rename. > > > > > > Here's my question, how to do this when searches are running live? > > > > > > Say I have a directory that holds the current valid index: > > > > > > C:\myindex > > > > > > and when I'm running my nightly process to generate the index, it gets > > > temporarily indexed to: > > > > > > C:\temp\myindex > > > > > > How can I very quickly replace C:\myindex with C:\temp\myindex? > > > > > > I can't simply do a rename since C:\myindex will likely have open > > > files. (Gotta love windows) > > > > > > And I can't delete all files in myindex, again because of the open files issue. > > > > > > Any ideas? > > > > > > Thanks, > > > Patrick > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
