"Yonik Seeley" <[EMAIL PROTECTED]> wrote: > > How can this lead to index corruption? The "no such file or directory" on > > loading _cf9.fnm sounds like index corruption? > > I don't think older versions of lucene handled these errors as well. > Perhaps _cf9.fnm failed to be written, but the segments file succeeded. > It could also be Solr's fault for allowing further operations on an > index after one failed? I'm not sure how that should be handled.
Ahh, OK. Yeah it's not clear how Solr should handle this case, though Lucene should be at (get to?) the point where on exception no harm to the index can be done. Ie, index on disk is left consistent, and, the state of the writer is such that it can't corrupt the index even if it's further used after the exception. > > Or maybe the index is not corrupt but then we are hitting the descriptor > > limit > > on opening a searcher and it's being reported as "no such file or > > directory"? > > Hmmm, yes that's possible too... > Should be easy to tell by checking if the file _cf9.fnm already exists. Oh yeah. Ryan, does that file exist? > > Yonik do you understand why so many unreferenced files are being produced > > here? What's the root cause? > > Just guesses... but perhaps new index files are written but things > fail before the stage where old files are deleted? OK, though, writer doesn't use many descriptors at all, right? It opens 1 segment's worth to flush, and mergeFactor+1 segment's worth to merge. It's spooky if Lucene can create zillions of un-referenced files. Would be good to get to the root cause & make sure it's really fixed on trunk. Or, maybe they are all referenced -- I think there were issues with the old merge policy that could cause segments to not be merged when they should have been? Mike --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
