This is going to sound stupid, but my problem was in fact much simpler than a complex sync bug. The previous bug report was a red herring for my problem. As it turns out, several indices was recently moved to this new server. Most of the indices were tight merged, however, as I discovered, the ones causing problems were not. The service running Lucene did not have modify/delete privileges on the directory and so when the service started and tried to merge some of the indices, the merge failed. Worse yet, my app caught the exception, logged it, only to try it again at a later time -- causing more data to be written to disk and if left unchecked, the server would run out of space because it kept trying to merge the files.
In retrospect, it would have been nice if a SecurityException, or the like, was thrown instead of a generic IOException. It was extremely long frustrating not knowing the underlying cause of the file rename error, insufficient OS security privileges. Is there a way to clarify this error using the OS level access information? - Jeff > -----Original Message----- > From: Vanlerberghe, Luc [mailto:[EMAIL PROTECTED] > Sent: Wednesday, January 25, 2006 7:57 AM > To: [email protected] > Subject: RE: Optimize segment deletion exception > > I did a quick find in the current source and I saw two more > functions that access the "segments" file without using a commit lock. > IndexReader.lastModified(File directory) and > IndexReader.lastModified(Directory directory) > > Perhaps your code uses one of those to check if the index is > current instead of isCurrent() or getCurrentVersion() as documented? > > This would mean that even calling File.lastModified() can > prevent Windows from deleting a file... > > Luc > > > -----Original Message----- > From: Dalton, Jeffery [mailto:[EMAIL PROTECTED] > Sent: dinsdag 24 januari 2006 21:16 > To: [email protected] > Subject: Optimize segment deletion exception > > I have a class which uses the Lucene 1.9 (Jan 24th svn > version) IndexModifier to write documents, delete documents, etc... > Periodically, I have a scheduled task which uses this > IndexModifier to optimize the index. I've been tracking down > a problem/bug. I can re-produce it pretty consistently > running my program. I simply open an IndexModifier wait, > without adding documents, and then trying to optimize. Doing > this I sometimes get the following error: > > java.io.IOException: Cannot delete C:\01570_01579\segments > at > org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:265) > at > org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:95) > at > org.apache.lucene.index.IndexWriter$4.doBody(IndexWriter.java:698) > at org.apache.lucene.store.Lock$With.run(Lock.java:109) > at > org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter. > java:702) > at > org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter. > java:658) > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:517) > at > org.apache.lucene.index.IndexModifier.optimize(IndexModifier.java:328) > > I have seen this fail for an index repeated times in a row, > although I haven't tested to see if the same indices cause > trouble across restarting the program. > > Re-reading this mailing list, I found the following bug > report submitted and patched by Luc (via Yonik): > http://issues.apache.org/jira/browse/LUCENE-481 regarding the > IndexReader on Dec 22nd. > > After reading this, I upgraded to the latest Lucene source, > which I got from SVN today(Jan 24th). This includes Luc's > patch submitted on Dec 22nd. The above error is from the > latest, patched, code. > > I am running on Windows Server 2003, Web edition with the > latest 1.4.2 Sun JDK. This seems a bit bizarre, is there a > windows platform problem? > When the IndexModifier closes the IndexReader, are some > resources not being released? > > The following code in FSDirectory is throwing the error: > if (nu.exists()) > if (!nu.delete()) > throw new IOException("Cannot delete " + nu); > > Clearly, deleting the segs file is failing. Another process > (a reader?) still has an open handle? > > I've got to get this fixed, so I'll do whatever is necessary. > Any known issues or tips/assistance tracking this one down > would be greatly appreciated. If there is any other > information I can provide to > troubleshoot, I'll try and get it done. I'm working on a > test case to > try and see if I can isolate the problem. > > Thanks, > > - Jeff > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
