Re: NRT consistency

2011-04-11 Thread Mark Miller
- Amazon Dynamo uses vector clocks for this. > > Otis > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > - Original Message >> From: Mark Miller >> To: java-user@lucene.

Re: NRT consistency

2011-04-11 Thread Mark Miller
On Apr 11, 2011, at 1:05 PM, Em wrote: > Thank you both! > > Mark, could you explain what you mean? I never heard from such an > index-splitter. BTW: The idea of having a segment per document sounds a lot > like an exception for too many FileDescriptors :) This is just an idea for rebalancing I

Re: NRT consistency

2011-04-11 Thread 张成
Something like dynamo's pattern, in the near real time searching, we should make N = W. 在 2011 4 11 23:52,"Mark Miller" 写道: > > On Apr 10, 2011, at 4:34 AM, Em wrote: > >> Hello list, >> >> I am currently trying to understand Lucene's Near-Real-Time-Feature which >> was covered in "Lucene in Actio

Re: NRT consistency

2011-04-11 Thread Otis Gospodnetic
Miller > To: java-user@lucene.apache.org > Sent: Mon, April 11, 2011 11:52:05 AM > Subject: Re: NRT consistency > > > On Apr 10, 2011, at 4:34 AM, Em wrote: > > > Hello list, > > > > I am currently trying to understand Lucene's Near-Real-Time-

Re: NRT consistency

2011-04-11 Thread Michael McCandless
On Mon, Apr 11, 2011 at 1:05 PM, Em wrote: > Mike, as you said, the segments are flushed like normal. > Let's say my server dies for whatever reason, when restarting it and > reopening the index-writer: Does the IW deletes the flushed file, because it > is not mentioned in the segmentInfo - file

Re: NRT consistency

2011-04-11 Thread Em
Thank you both! Mark, could you explain what you mean? I never heard from such an index-splitter. BTW: The idea of having a segment per document sounds a lot like an exception for too many FileDescriptors :) Mike, as you said, the segments are flushed like normal. Let's say my server dies for wha

Re: NRT consistency

2011-04-11 Thread Mark Miller
On Apr 10, 2011, at 4:34 AM, Em wrote: > Hello list, > > I am currently trying to understand Lucene's Near-Real-Time-Feature which > was covered in "Lucene in Action, Second Edition". > > Let's say I got a distributed system with a master and a slave. > > In Solr replication is solved by check

Re: NRT consistency

2011-04-10 Thread Michael McCandless
That's a neat question (how to replicate index changes from opening a new NRT reader). The good news, the segments are flushed "like normal", so they can be replicated to the mirror(s). But the bad news is, the segments file is not written to disk -- it's held only in RAM (inside IW and shared to