Thanks for the response, Mike. If I understand correctly, the problem was incorrectly identifying a large corrupted index as a non-existant index? It seems like you'd really want an index with first-commit in progress to behave like an index with zero documents, as opposed to a non-existant index. That is, if segments_1 is corrupt, open an empty index. This seems more analagous to how lucene treats pre-committed segments_n files after the first. I don't know how difficult it would be to implement, in practice.
This isn't a major issue for us either way. We can work around it by having our searcher code check for the existence of a segments.gen and just return zero hits by default. Since this is a read path only, we don't need to worry about incorrectly overwriting the index. Cheers, Geoff On Thu, May 16, 2013 at 1:47 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > Unfortunately this is expected behavior. > > We tried to fix it in LUCENE-2812, but this fix was too dangerous and > could sometimes erase a good index (if transient IOExcs are happening, > e.g. due to file descriptor exhaustion) so we reverted back in > LUCENE-4738, so that indexExists will return true, and trying to open > an IndexReader will throw CorruptIndexException, when the first-commit > has started but not yet finished. > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, May 16, 2013 at 10:45 AM, Geoff Cooney <cooney.ge...@gmail.com> > wrote: > > Hi, > > > > We're occasionally seeing a CorruptIndexException when a searcher is > opened > > on a new index. > > > > When we see the exception, it looks like what is happening is that the > > searcher is opening the index after prepareCommit for segments_1 but > before > > the commit is completed. Because there is no prior commit to fall back > to, > > lucene throws the CorruptIndexException. Does this sounds feasible and > is > > it a bug? My expectation is that lucene would treat the index as > > nonexistent until the first commit is completed. > > > > We're running lucene 3.5. > > > > Cheers, > > Geoff > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
