I've made some changes to my Solr setup, and now I'm getting the
"background merge hit exception" pasted at the end of this message.
The most notable changes I've made are:
Update to r690989 (Lucene r688745)
Change a few things in my schema. In particular, I was previously
storing my main document text and the metadata fields in a single
"body" field, like this:
*****
<!-- This is what we use for the default text search and for
highlighting : -->
<field name="body" type="shingleText" indexed="true" stored="true"
omitNorms="false" multiValued="true" />
<copyField source="companyname" dest="body"/>
<copyField source="date" dest="body"/>
*****
whereas I'm now using "body" as a sort of alias that just gets
redirected to other fields, like this:
*****
<!-- This is only an alias used to populate other fields using copyField. -->
<field name="body" type="shingleText" indexed="false" stored="false" />
<!-- This is what we use for the default text search: -->
<field name="everything" type="shingleText" indexed="true"
stored="false" omitNorms="false" multiValued="true" />
<!-- Stored text for use with highlighting -->
<field name="kwic" type="shingleText" indexed="false" stored="true"
compressed="true" multiValued="false" />
<!-- Note: The maxLength parameter is for the copyField max length
patch (SOLR-538). At first I thought
that patch was causing my difficulties, but then
I had the same trouble when I built without that patch -->
<copyField source="body" dest="kwic" maxLength="100000" />
<copyField source="body" dest="everything"/>
<copyField source="companyname" dest="everything"/>
<copyField source="date" dest="everything"/>
*****
When I was indexing with this new setup, things were initially fine,
and segments seemed to be merging fine. I ran into trouble when I sent
an <optimize/>, though. I think in an earlier run I also got a very
similar exception just from document adds, without an explicit
<optimize/>.
I'm also running with a shingle-related patch
(https://issues.apache.org/jira/browse/LUCENE-1370 /
https://issues.apache.org/jira/browse/SOLR-744) and the rich document
handler patch, though I've used these before without trouble.
Is it possible that my schema change is illegitimate? Am I not allowed
to have non-indexed, non-stored fields, for example?
Anyway, here is my stack trace:
*****
background merge hit exception: _1h:C2552 _1i:C210->_1i _1j:C266->_1i
_1k:C214->_1i _1l:C329->_1i _1m:C231->_1i _1n:C379->_1i _1o:C447
_1p:C453->_1p _1q:C485->_1p into _1r [optimize]
java.io.IOException: background merge hit exception: _1h:C2552
_1i:C210->_1i _1j:C266->_1i _1k:C214->_1i _1l:C329->_1i _1m:C231->_1i
_1n:C379->_1i _1o:C447 _1p:C453->_1p _1q:C485->_1p into _1r [optimize]
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2303)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2233)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:355)
at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:77)
at
org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:228)
at
org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:125)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1156)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.lucene.store.BufferedIndexOutput.writeBytes(BufferedIndexOutput.java:49)
at
org.apache.lucene.index.FieldsWriter.writeField(FieldsWriter.java:215)
at
org.apache.lucene.index.FieldsWriter.addDocument(FieldsWriter.java:268)
at
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:359)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:138)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3988)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3637)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:214)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:269)
