Well ... I was just thinking that today, because of stale caches, we might read segments_5 from either segments.gen or directory listing, and attempt to read and succeed. Then the reader will see the index 'until segment 5'.
So when I analyze the exception I think that: * The directory listing couldn't return segments_8 as candidate, as otherwise we should have succeeded reading it? * It might have been returned from segments.gen, but since the directory listing cache is stale, the client thinks the file does not exist. ** At which point we'd attempt to read segments_7, but may fail again. So if afterwards we read until segment_17 and exhaust read-ahead, and we determine that there's a problem - we throw the exception. If instead we'll try to read backwards, I'm sure one of the segments will be read successfully, because that reader must already see any segment, right? Or am I completely off here? Shai On Fri, Aug 14, 2009 at 12:26 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > On Thu, Aug 13, 2009 at 5:02 PM, Shai Erera<ser...@gmail.com> wrote: > > What about doing a read-backwards as well? I.e., if read ahead fails, try > to > > read backwards --> we must be able to read any segment, no? > > How would that help? > > Ie, IndexWriter only writes "forwards". So if a filesystem cache is > stale, it must be the case that the "truth" lies forwards? Ie a cache > would never lie, saying the segmetns_N file was bigger than it really > is? > > Mike > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > >
