Re: IndexWriter.getReader usage

2009-08-03 Thread DM Smith
On 08/03/2009 08:21 AM, Earwin Burrfoot wrote: The biggest win for NRT was switching to per-segment Collector because that meant we could re-use FieldCache entries for all segments that hadn't changed. In my opinion, this switch was enough to get as NRT-ey, as you want. Fusing IR/IW togeth

Re: IndexWriter.getReader usage

2009-08-03 Thread Earwin Burrfoot
> The biggest win for NRT was switching to per-segment Collector because > that meant we could re-use FieldCache entries for all segments that > hadn't changed. In my opinion, this switch was enough to get as NRT-ey, as you want. Fusing IR/IW together makes Lucene a great deal more complicated and

Re: IndexWriter.getReader usage

2009-08-02 Thread Michael McCandless
It's likely the RAMDir approach will still be a performance win over getReader, until LUCENE-1313 is in (which uses a RAMDir for the small-enough newly flushed segments). getReader() writes (but doesn't sync) the new segment files to the Directory, and then opens a new SegmentReader on those files

Re: IndexWriter.getReader usage

2009-08-01 Thread Grant Ingersoll
On Aug 1, 2009, at 8:45 AM, Yonik Seeley wrote: On Sat, Aug 1, 2009 at 8:30 AM, DM Smith wrote: On Aug 1, 2009, at 7:52 AM, Grant Ingersoll wrote: I'm wondering how the new IndexWriter.getReader stuff relates to that approach? Is there even a need for the RAM dirs at this point? I'm cur

Re: IndexWriter.getReader usage

2009-08-01 Thread Yonik Seeley
On Sat, Aug 1, 2009 at 8:30 AM, DM Smith wrote: > On Aug 1, 2009, at 7:52 AM, Grant Ingersoll wrote: >> I'm wondering how the new IndexWriter.getReader stuff relates to >> that approach?  Is there even a need for the RAM dirs at this point? > > I'm curious as to how it obviates the need for a RAM

Re: IndexWriter.getReader usage

2009-08-01 Thread DM Smith
On Aug 1, 2009, at 7:52 AM, Grant Ingersoll wrote: In many NRT cases, it seems the traditional approach has been to have two RAM directories and a write-through FS Directory (for example Zoie does this, and it has also been discussed a fair number of times on the various lists). I'm wonde