On Fri, Apr 3, 2009 at 5:42 PM, Jason Rutherglen <jason.rutherg...@gmail.com> wrote: >> I think the realtime reader'd just store the maxDocID it's allowed to >> search, and we would likely keep using the RAM format now used. > > Sounds pretty good. Are there any other gotchas in the design?
Yes: the flushing process becomes challenging. When we flush, we must forcefully cutover any open readers searching that RAM buffer, to the now-on-disk segment. Such swap-out is tricky because there could be [many threads of] searches in-flight, iterating through postings, etc. Which means the RAM buffer would have to become an independent entity that is not re-used after flushing, but instead sticks around until GC determines all outstanding readers have switched to the on-disk segment. I would rather not go here unless it's clear the current near real-time performance is too limiting. But my simplistic test the current performance looks good. And, if the performance does turn out to be lacking, the next step (before searching IW's ram buffer) is to flush the new tiny segments through a RAMDir, first. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
