Hi Vitaly, This is unfortunately due to a recent change in IndexWriter ... search for the recent thread "CorruptIndexException when opening Index during first commit" on this list.
Normally, if a commit fails for any reason (OS, hardware, JVM crashes) we simply fall back to the last commit and no intervention is necessary. But if that commit was the very first commit ... we used to try to detect this and treat it as if there were no index present. This is the ideal/correct behavior, because "no index present" was in fact the state of the index before this failed commit. However that logic was dangerous and would sometimes result in IndexWriter thinking no index was present when in fact there was one, when there were transient errors e.g. due to file descriptor exhaustion. So to be safe we now throw the CorruptIndexException on first commit back to the application. Your workaround sounds good. Another option instead of the separate init file, on hitting a CorruptIndexException, might be to list the directory: if there is only one file, segments_1, then it's a corrupt first commit and you can recreate the index. Mike McCandless http://blog.mikemccandless.com On Wed, May 29, 2013 at 8:09 PM, Vitaly Funstein <vfunst...@gmail.com> wrote: > I have encountered a strange issue, that appears to be pretty hard to hit, > but is still a serious problem when it does occur. It seems that if the JVM > crashes in a racy fashion with instantiation of IndexWriter, the index may > be left in an inconsistent state. An attempt to reload such an index on > restart will result in a CorruptIndexException thrown when calling the > constructor again. > > My idea of a workaround involves the following: > > - call commit() immediately after invoking the constructor. > - create a special init file after the commit to indicate the index was > created successfully > - on app startup, check to see if the init file is there; if not, the index > is corrupt and needs to be recreated. > > (The last two already happen in my code) > > Is this a valid approach to protect from this sort of race? Is there > already a built-in mechanism to make constructor fully synchronous wrt > persisting all the metadata? If not, it seems like there probably should > be... --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
