You're catching an exception and acting on it, but you're not reporting it, for now, comment out the deletion and copyfrombackup and try reporting errors, if the batch is failing on a regular basis you want to know about it, comment the deletion and copy across code out, also watch out that if you backup the index during an indexing you could end up with a limp index missing a few files, hence the missing segments, I would check for write and commit locks pre-backup so as to avoid that. This is probably caused by two unrelated errors first batchindex() fails then the backup restores a version that may not have all the indexes there (depending when it was backed up) thereby giving you the feeling that segments are disappearing randomly.
Hope this helps. Nader Henein -----Original Message----- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Monday, May 03, 2004 6:52 AM To: Lucene Users List Subject: RE: Disappearing segments Thanks for responding Nader. hmmmm...you've hit the nail on the spot. I do have a cron job which backs up the index. Its run in a batch index scheduled job. The logic is basically backupindex() try { batchindex() } catch(Exception e) { deleteindex(); copyfrombackuptoindex() deletebackup(); } I assume that the original index before backing up was complete and 'working'. I'm also deleting the index that failed, instead of just overwriting. Where did I go wrong? I'm not checking that the index isn't write-locked before backing up, but I don't think that's the problem (though it very well can be a separate problem). Kelvin On Fri, 30 Apr 2004 23:20:42 +0400, Nader Henein said: > Could you share you're indexing code, and just to make sure id there > anything running on your machine that could delete these files, like > an a cron job that'll back up the index. > > You could go by process of elimination and shut down your server and > see if the files disappear, coz if the problem is contained within the > server you know that you can safely go on the DEBUG rampage. > > Nader > > -----Original Message----- > From: Kelvin Tan [mailto:[EMAIL PROTECTED] > Sent: Friday, April 30, 2004 9:15 AM > To: Lucene Users List > Subject: Re: Disappearing segments > > An update: > > Daniel Naber suggested using IndexWriter.setUseCompoundFile() to see > if it happens with the compound index format. Before I had a chance to > try it out, this happened: > > java.io.FileNotFoundException: C:\index\segments (The system cannot > find the file specified) at java.io.RandomAccessFile.open(Native > Method) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200) > at > org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.j > ava:321) > at > org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:329) > at > org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:268) > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:71) > at > org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:154) > at org.apache.lucene.store.Lock$With.run(Lock.java:116) > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:149) > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:131) > > so even the segments file somehow got deleted. Hoping someone can shed > some light on this... > > Kelvin > > On Thu, 29 Apr 2004 11:45:36 +0800, Kelvin Tan said: >> Errr, sorry for the cross-post to lucene-dev as well, but I realized >> this mail really belongs on lucene-user... >> >> I've been experiencing intermittent disappearing segments which >> result in the following stacktrace: >> >> Caused by: java.io.FileNotFoundException: C:\index\_1ae.fnm (The >> system cannot find the file specified) at >> java.io.RandomAccessFile.open(Native Method) at >> java.io.RandomAccessFile.<init>(RandomAccessFile.java:200) >> at >> org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.j >> a >> va:321) at >> org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:329) >> at org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:268) >> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:78) >> at >> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:104) >> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:95) >> at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:112) >> at org.apache.lucene.store.Lock$With.run(Lock.java:116) >> at org.apache.lucene.index.IndexReader.open(IndexReader.java:103) >> at org.apache.lucene.index.IndexReader.open(IndexReader.java:91) >> at >> org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:75) >> >> The segment that disappears (_1ae.fnm) varies. >> >> I can't seem to reproduce this error consistently, so don't have a >> clue what might cause it, but it usually happens after the >> application has been running for some time. Has anyone experienced >> something similar, or can anyone point > me >> in the right direction? >> >> When this occurs, I need to rebuild the entire index for it to be >> usable. Very troubling indeed... >> >> Kelvin >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
