Hi Yonik, Thanks for the response.
reply inline. On Tue, Dec 16, 2008 at 6:44 AM, Yonik Seeley <ysee...@gmail.com> wrote: > On Tue, Dec 16, 2008 at 1:04 AM, Michael Stoppelman <stop...@gmail.com> > wrote: > > I've got a question from Doug's original email about replication ( > > http://www.mail-archive.com/lucene-u...@jakarta.apache.org/msg12709.html > ): > > > > "1. On the index master, periodically checkpoint the index. Every minute > or > > so the IndexWriter is closed and a 'cp -lr index index.DATE' command is > > executed from Java, where DATE is the current date and time. This > > efficiently makes a copy of the index when its in a consistent state by > > constructing a tree of hard links. If Lucene re-writes any files (e.g., > the > > segments file) a new inode is created and the copy is unchanged." > > > > Is closing the IndexWriter really a requirement on taking a snapshot? Or > can > > one take a snapshot on an index being written, I've done this in my > > development environment and it seems to work fine w/o closing the > > IndexWriter. > > There are subtle race conditions if you try to do this with a changing > index. > At any instance in time, the index should be consistent, *but* you > can't actually make a snapshot instantaneously. Is the race condition in writing out the segments.gen or segments_N files? >From my understanding index segments once closed by the IndexWriter they aren't modified again (they might be deleted though if they're merged away). > > So this is doable, but it would require some complex retry logic like > IndexReader has when opening an index. > > > Also the solr replication shell scripts don't seem to worry > > about this either. > > Solr takes snapshots when it knows it's not updating the index (new > index changes are internally blocked when calling snapshooter). > > -Yonik > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
