Hi Mike, Yes, we had OS crash but it happened minutes after we commited/closed IndexWriter according to our log file. We will consider lucene upgrade.
Thanks very much for helps, Lisheng -----Original Message----- From: Michael McCandless [mailto:luc...@mikemccandless.com] Sent: Saturday, October 29, 2011 10:55 AM To: java-user@lucene.apache.org Subject: Re: data corruption in lucene index 2.3.2 That's fine -- the reader is read-only and won't corrupt the index if the machine/OS crashes while it's open. Oh, actually: Lucene 2.3.x did not properly fsync files when you closed the IndexWriter (this was fixed in 2.4.0). This means even if you close the writer and a crash occurs the index could become corrupt. Did you have an OS/machine crash on this index? Mike McCandless http://blog.mikemccandless.com On Sat, Oct 29, 2011 at 12:15 PM, Zhang, Lisheng <lisheng.zh...@broadvision.com> wrote: > Hi Mike, > > Thanks very much for helps, if indexer was closed but IndexSearcher > (therefore IndexReader) was open when there was OS crash, could that > cause segment data corruption (I guess it should not)? > > Best regards, Lisheng > > -----Original Message----- > From: Michael McCandless [mailto:luc...@mikemccandless.com] > Sent: Saturday, October 29, 2011 3:39 AM > To: java-user@lucene.apache.org > Subject: Re: data corruption in lucene index 2.3.2 > > > Was there any catastrophic event against this index? > > EG, power loss on the machine, or OS crash, while the app had and > IndexWriter open? > > Mike McCandless > > http://blog.mikemccandless.com > > On Fri, Oct 28, 2011 at 4:57 PM, Zhang, Lisheng > <lisheng.zh...@broadvision.com> wrote: >> >> We are using lucene 2.3.2 (yes we should upgrade) and recently we had >> Exception when opening >> index: >> >> ### >> java.io.IOException: read past EOF<?xml:namespace prefix = o ns = >> "urn:schemas-microsoft-com:office:office" /> >> at >> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:146) >> at >> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38) >> at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:66) >> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:207) >> at >> org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexReader.java:68) >> at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:636) >> at >> org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.java:63) >> at org.apache.lucene.index.IndexReader.open(IndexReader.java:209) >> at org.apache.lucene.index.IndexReader.open(IndexReader.java:173) >> at >> org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:48) >> ### >> >> Looking at the data I found segment files having zero bytes: >> >> ### >> -rwxrwxrwx 1 root root 0 2011-10-29 01:35 segments_8tbb >> -rwxrwxrwx 1 root root 0 2011-10-29 01:35 segments.gen >> ### >> >> How could that happen (I did not find any multiple indexing on this data >> folder)? >> >> Thanks very much for helps, Lisheng >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org