Re: Per-Thread DW and IW

2010-04-24 Thread Doron Cohen
On Thu, Apr 22, 2010 at 5:04 PM, Michael McCandless luc...@mikemccandless.com wrote: I like the slice term, but, can we drop the 'd'? Ie SliceWriter and SliceReader. I agree, it is better without the 'd'.

Re: Per-Thread DW and IW

2010-04-23 Thread Shai Erera
The big picture includes what you write, but also other usage, such as loading different slices into memory, introduce the complementary API to ParallelReader, query a single slice only etc. Shai On Thu, Apr 22, 2010 at 5:04 PM, Michael McCandless luc...@mikemccandless.com wrote: I like the

Re: Per-Thread DW and IW

2010-04-21 Thread Michael Busch
Yeah, sounds like we have the same things in mind here. In fact, this is pretty similar to what we discussed a while ago on LUCENE-2026 I think. SegmentWriter could be a higher level interface with more than one implementation. E.g. there could be one SegmentWriter that supports appending

Re: Per-Thread DW and IW

2010-04-21 Thread Shai Erera
I don't advocate to develop PI as an external entity to Lucene, you've already done that ! :) We should open up IW enough to develop PI efficiently, but I think we should always allow some freedom and flexibility to using applications. If IW simply created a Parallel DW, handle the merges on its

Re: Per-Thread DW and IW

2010-04-20 Thread Marvin Humphrey
On Tue, Apr 20, 2010 at 07:27:57AM -0400, Michael McCandless wrote: There are elements of IW that still must be centralized -- managing the merge policy/schedulers, deletion policy, writing/committing the segments files, managing ongoing addIndexes, tracking pending deletions, the reader pool,