[ http://issues.apache.org/jira/browse/LUCENE-425?page=comments#action_12426910 ] Michael McCandless commented on LUCENE-425: -------------------------------------------
There was a recent thread on java-user that looks very similar to this issue: http://www.gossamer-threads.com/lists/lucene/java-user/38669 When Lucene writes a new "segments" or new "deletable" file it first writes to X.new and then renames X.new --> X. Apparently that renameTo can sometimes fail, even though both the X.new and the X file are "closed" from Java. The "best guess" from that discussion is that there is a JVM bug on certain JVMs on Windows, whereby a file handle (at the Windows OS level) is held open even after from Java it has been "closed". Ant has a workaround for this issue: force a GC, wait, and then retry. I don't really like that workaround for Lucene because we should not suddenly / unexpectedly do a system level action like forcing GC -- it's a "leaky abstraction" (http://www.joelonsoftware.com/articles/LeakyAbstractions.html). Which JVM are you using? I couldn't reproduce this; I made a small change to the IndexFiles demo command-line tool and then refreshed the view from Windows Explorer. On Sun JDK 1.4.2_10 and Sun JDK 1.5.0_07 I couldn't get the IOException. Issue LUCNE-554 also proposes more robust logic around opening the .NEW file if it exists. I think that makes alot of sense. > "Cannot rename segments.new to segments" > ---------------------------------------- > > Key: LUCENE-425 > URL: http://issues.apache.org/jira/browse/LUCENE-425 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 1.4 > Environment: Operating System: Windows XP > Platform: PC > Reporter: Mikko Noromaa > Assigned To: Lucene Developers > > AddDocument periodically fails with "Cannot rename segments.new to segments". > Sometimes it happens after 10,000 documents, sometimes after 200,000 > documents. > My index is placed on a FAT partition. My Lucene version is 1.4.3. I am using > a single IndexWriter object and doing all AddDocuments calls serially from a > single thread. > I saw some discussion about this problem in the mailing lists, but looks like > the problem wasn't completely eliminated yet. > My suggested solutions are as follows: > - If renameTo() fails, wait some time and retry. After this, apply the other > methods that are now in FSDirectory.java. Retry them as well after waiting > for > some time. > - In the IndexWriter constructor, check for 'segments.new' if 'segments' > cannot be found. This would allow recovery if FSDirectory failed on a > previous > run. Now I have to write the recovery code myself outside of Lucene. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
