Hmm... somehow Lucene is flushing a new segment on closing the IndexWriter, and thinks 1 doc had been added to the stored fields file, yet the fdx file is the wrong size (0 bytes). This check (& exception) are designed to prevent corruption from entering the index, so it's at least good to see CheckIndex passes after this.
I don't think you're hitting LUCENE-1521: that issue only happens if a single segment has more than ~268 million docs. Which exact JRE version are you using? When you hit this exception, is it always "1 docs vs 0 length in bytes"? Mike On Wed, May 20, 2009 at 3:19 AM, James X <hello.nigerian.spamm...@gmail.com> wrote: > Hello all,I'm running Solr 1.3 in a multi-core environment. There are up to > 2000 active cores in each Solr webapp instance at any given time. > > I've noticed occasional errors such as: > SEVERE: java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs > vs 0 length in bytes of _h.fdx > at > org.apache.lucene.index.StoredFieldsWriter.closeDocStore(StoredFieldsWriter.java:94) > at > org.apache.lucene.index.DocFieldConsumers.closeDocStore(DocFieldConsumers.java:83) > at > org.apache.lucene.index.DocFieldProcessor.closeDocStore(DocFieldProcessor.java:47) > at > org.apache.lucene.index.DocumentsWriter.closeDocStore(DocumentsWriter.java:367) > at > org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:567) > at > org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3540) > at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450) > at > org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1638) > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1602) > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1578) > at > org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:153) > > during commit / optimise operations. > > These errors then cause cascading errors during updates on the offending > cores: > SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed > out: SingleInstanceLock: write.lock > at org.apache.lucene.store.Lock.obtain(Lock.java:85) > at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1070) > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:924) > at > org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:116) > at > org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:122) > > This looks like http://issues.apache.org/jira/browse/LUCENE-1521, but when I > upgraded Lucene to 2.4.1 under Solr 1.3, the issue still remains. > > CheckIndex doesn't find any problems with the index, and problems disappear > after an (inconvenient, for me) restart of Solr. > > Firstly, can I as the symptoms are so close to those in 1521, can I check my > Lucene upgrade method should work: > - unzip the Solr 1.3 war > - remove the Lucene 2.4dev jars > (lucene-core, lucene-spellchecker, lucene-snowball, lucene-queries, > lucene-memory,lucene-highlighter, lucene-analyzers) > - move in the Lucene 2.4.1 jars > - rezip the directory structures as solr.war. > > I think this has worked, as solr/default/admin/registry.jsp shows: > <lucene-spec-version>2.4.1</lucene-spec-version> > <lucene-impl-version>2.4.1 750176 - 2009-03-04 > 21:56:52</lucene-impl-version> > > Secondly, if this Lucene fix isn't the right solution to this problem, can > anyone suggest an alternative approach? The only problems I've had up to now > is to do with the number of allowed file handles, which was fixed by > changing limits.conf (RHEL machine). > > Many thanks! > James >
