One question: do you know whether 67,861 docs "feels like" a newly flushed segment, or, the result of a merge?

Ie, roughly how many docs are you buffering in IndexWriter before it flushes? Are they very small documents and your RAM buffer is large?

Mike

Ian Lea wrote:

Hi


When bulk loading into a new index I'm seeing this exception

Exception in thread "Thread-1"
org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: doc counts differ for
segment _4l: fieldsReader shows 67861 but segmentInfo shows 67862
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run (ConcurrentMergeScheduler.java:271)
Caused by: org.apache.lucene.index.CorruptIndexException: doc counts
differ for segment _4l: fieldsReader shows 67861 but segmentInfo shows
67862
at org.apache.lucene.index.SegmentReader.initialize (SegmentReader.java:313)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
at org.apache.lucene.index.IndexWriter.mergeMiddle (IndexWriter.java:3093)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run (ConcurrentMergeScheduler.java:240)

when use java version 1.6.0_05-b13 or 1.6.0_04-b12 on linux, with
lucene 2.3.0 or 2.3.1 or lucene-core-2.3-SNAPSHOT from yesterday.

With java version 1.6.0_03-b05 things work fine.

The exception happens a few hundred thousand documents into the load.

A different program updating a different index with different data on
a different server gave a similar error on version 1.6.0_05-b13 and
lucene 2.3.0.


Any ideas? Is this maybe a known issue or am I missing something obvious?



--
Ian.

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to