On Nov 10, 2007 4:34 PM, Michael McCandless <[EMAIL PROTECTED]> wrote: > > "Yonik Seeley" <[EMAIL PROTECTED]> wrote: > > On Nov 10, 2007 4:01 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote: > > > Using solr, we have been running an indexing process for a while and > > > when I checked on it today, it spits out an error: > > > > > > java.lang.RuntimeException: java.io.FileNotFoundException: > > > /path/to/index/_cf9.fnm (No such file or directory) > > > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:584) > > > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:475) > > > > > > Looking through the archives, it looks like we are up a creek. > > > > > > Any thoughts on what could have caused this? The log files contains > > > some 'too many open files' errors, I can't tell if that corresponds with > > > when the index went bad though. > > > > Yup... that would most likely be it. > > How can this lead to index corruption? The "no such file or directory" on > loading _cf9.fnm sounds like index corruption?
I don't think older versions of lucene handled these errors as well. Perhaps _cf9.fnm failed to be written, but the segments file succeeded. It could also be Solr's fault for allowing further operations on an index after one failed? I'm not sure how that should be handled. > Or maybe the index is not corrupt but then we are hitting the descriptor limit > on opening a searcher and it's being reported as "no such file or directory"? Hmmm, yes that's possible too... Should be easy to tell by checking if the file _cf9.fnm already exists. > If so, maybe opening an IndexWriter and optimizing the index would recover > it? (Make a backup copy of the index first!). > > > > 2. Does the ulimit number explain how the index got corrupted? If so, > > > it seems like a problem. > > > > I think newest lucene versions would prevent this with > > lucene_autocommit=false. A new segments file (the file which > > references all other files in the current index) is not written until > > a close of the writer. > > Yonik do you understand why so many unreferenced files are being produced > here? What's the root cause? Just guesses... but perhaps new index files are written but things fail before the stage where old files are deleted? Ryan, what version of lucene is this version of Solr using? -Yonik --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
