Ya know, It turned out to be embarrassingly simple - I think I just had a mental block from thinking about how Solr's warming worked for so long.
Actually, it was so simple, yet I still got in wrong on the first glance, that it reminded me of this: http://www.marilynvossavant.com/forum/viewtopic.php?t=64 fun stuff ;-) -Yonik http://www.lucidimagination.com On Tue, Sep 22, 2009 at 1:42 PM, Michael McCandless <luc...@mikemccandless.com> wrote: > Well described, that's exactly it! I like the concrete example :) > > Thanks Yonik. > > Mike > > On Tue, Sep 22, 2009 at 1:38 PM, Yonik Seeley > <yo...@lucidimagination.com> wrote: >> OK Mike, thanks for your patience - I understand now :-) >> >> Here's an example that helped me understand - hopefully it will add to >> others understanding more than it confuses ;-) >> >> IW.getReader() => segments={A, B} >> // something causes a merge of A,B into AB to start >> addDoc(doc1) >> // doc1 goes into segment C >> IW.getReader() => segments={A, B, C} >> // merge isn't done yet, so getReader() still returns A,B instead of >> AB, but doc1 is still searchable! >> >> OK, in this scenario, there's no advantage to warming in the IW vs the app. >> Let's start over with a little different timing: >> >> segments={A,B} >> // something causes a merge of A,B into AB to start >> addDoc(doc1) >> // doc1 goes into segment C >> // merging of A,B into AB finishes >> IW.getReader() => segments={AB, C} >> >> Oh, no... with warming at the app level, we need to warm the huge AB >> segment before doc1 is visible. We could continue using the old >> reader while the warming is ongoing, so no user requests will >> experience long queries, but doc1 isn't in the old segment. >> >> With warming in the IW (basically warming becomes part of the same >> operation as merging), then getReader() would return segments={A,B,C} >> and doc1 would still be instantly searchable. >> >> The only way to duplicate this functionality at the app layer would be >> to recognize that there is a new segment, try and figure out what old >> segments were merged to create this new segment, and create a reader >> that's a mix of old and new to avoid unwarmed segments - not nice. >> >> -Yonik >> http://www.lucidimagination.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
