Re: How can we know if 2 lucene indexes are same?

[Michael McCandless]

Shalin Shekhar Mangar wrote:

On Fri, Sep 5, 2008 at 6:03 PM, Michael McCandless <
[EMAIL PROTECTED]> wrote:


Large segment merges will also send huge traffic.  You may just  
want to
send all updates (document adds/deletes) to all slaves directly?   
It'd be
nice if you could somehow NOT sync the effects of segment merging,  
but do
sync doc add/deletes... not sure how to do that.


As Noble said, that is another option we can consider.

Well this is certainly a nice challenging problem :)

Thanks for the explanation Mike. The core problem is to make sure both
indices are in sync. The log replication helps us because we compare  
the
master and slave index with a reference point (log position). If it  
becomes
possible for us to specify a version number during a commit, we can  
use the
master's version number on the slave. This can help us compare the two
indices. Not sure if that API change will be generally useful.  
Thoughts?

I think this could be a generally useful feature?

So you're thinking IndexWriter.commit() would take an optional opaque  
argument (maybe a String for generality?) that's recorded into the  
segments_N and could then later be retrieved by IndexReader and  
IndexWriter?

After a merge completes, should it just carry forward whatever was  
stored on the last segments_N?

We should call it something other than version, which already exists  
-- maybe "commitDetails", "commitComment", "commitUserData" or  
something?

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]