On Jun 11, 2008, at 6:00 AM, Michael McCandless wrote:
Grant Ingersoll wrote:
Is more than one thread adding documents to the index?
I don't believe so, but I am trying to reproduce. I've only seen
it once, and don't have a lot of details, other than I noticed it
was on a specific file (.fdt) and was wondering if that was a
factor or not. That is, maybe Paul could reproduce it.
I think your exception differs from Paul's in an important way.
Paul's exception means an entire segment (its CFS file) was deleted,
which is very easily caused by accidentally allowing 2 writers on
the index at once. But in your case, SegmentReader successfully
opened the fnm file but then failed on the fdt, so, your segment
wasn't deleted (at least not entirely). So I think something
different caused your exception.
Any changes to the defaults in IndexWriter?
It's the SolrIndexWriter.
OK. But what does your solrconfig.xml look like?
It's pretty much the standard indexing setup that's on trunk. Merge
Factor 10, Max Buffered Docs 1000
After seeing that exception, does IndexReader.open also hit that
exception (ie, is/was the index corrupt)? Or does it only happen
with BG merges?
Not sure, unfortunately, I don't have a lot of info yet. The
background exception happened during an optimize, if that matters
at all
OK. It'd be very useful to know if index was really corrupt
(missing that segment) vs BG merge incorrectly, temporarily, thought
it was supposed to merge that segment.
Is this a largish index?
couple million relatively small records
Like, would there be so many segments that optimize would be
running concurrent merges (> 2*mergeFactor segments)? With
ConcurrentMergeScheduler, optimize is now able to run multiple
merges concurrently, if the index has enough segments.
I don't know. Like I said, the only reason I posted was b/c I thought
it sounded similar to Paul's. I haven't been able to reproduce it yet.
I'll run some stress tests, focusing on concurrency of merges during
optimize...
Which OS & JRE are you using?
Linux, Java 1.6.0_05
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
