When you hit corruption is it always this same problem?: java.lang.RuntimeException: term source:margolisphil docFreq=1 != num docs seen 0 + num docs deleted 0
Can you run with Lucene's IndexWriter infoStream turned on, and catch the output leading to the corruption? If something is somehow messing up the bits in the deletes file that could cause this. Mike On Mon, Jan 10, 2011 at 5:52 AM, Stéphane Delprat <stephane.delp...@blogspirit.com> wrote: > Hi, > > We are using : > Solr Specification Version: 1.4.1 > Solr Implementation Version: 1.4.1 955763M - mark - 2010-06-17 18:06:42 > Lucene Specification Version: 2.9.3 > Lucene Implementation Version: 2.9.3 951790 - 2010-06-06 01:30:55 > > # java -version > java version "1.6.0_20" > Java(TM) SE Runtime Environment (build 1.6.0_20-b02) > Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) > > We want to index 4M docs in one core (and when it works fine we will add > other cores with 2M on the same server) (1 doc ~= 1kB) > > We use SOLR replication every 5 minutes to update the slave server (queries > are executed on the slave only) > > Documents are changing very quickly, during a normal day we will have approx > : > * 200 000 updated docs > * 1000 new docs > * 200 deleted docs > > > I attached the last good checkIndex : solr20110107.txt > And the corrupted one : solr20110110.txt > > > This is not the first time a segment gets corrupted on this server, that's > why I ran frequent "checkIndex". (but as you can see the first segment is > 1.800.000 docs and it works fine!) > > > I can't find any "SEVER" "FATAL" or "exception" in the Solr logs. > > > I also attached my schema.xml and solrconfig.xml > > > Is there something wrong with what we are doing ? Do you need other info ? > > > Thanks, >
