What this probably indicates is that the size of the packets you send to Solr is large enough that it exceeds the transport protocol's limit. This is reinforced by your statement that reducing the batch size fixes the problem even though it increases indexing time.
So the place I'd be looking is the jetty configurations for any limits there. That said, what is your batch size? In my testing I pretty quickly get into diminishing returns, here's a writeup from some time ago: https://lucidworks.com/blog/2015/10/05/really-batch-updates-solr-2/ Best, Erick On Wed, Aug 17, 2016 at 12:03 PM, Jaspal Sawhney <jsawh...@sapient.com> wrote: > Bump ! > > On 8/16/16, 10:53 PM, "Jaspal Sawhney" <jsawh...@sapient.com> wrote: > >>Hello >>We are running solr 4.6 in master-slave configuration where in our master >>is used entirely for indexing. No search traffic comes to master ever. >>Off late we have started to get the early EOF error on the solr Master >>which results in a Broken Pipe error on the commerce application from >>where Indexing was kicked off from. >> >>Things to mention >> >> 1. We have a couple of sites each of which has the same document >>size but diff document count. >> 2. This error is being observed in the site which has the most number >>of document count I.e. 2204743 >> 3. The way I have understood solr to work is that irrespective of >>number of document the throughput is controlled by the ŒNumber of >>Threads¹ and ŒBatch size¹ - Am I correct? >> * In our case we have not touched the batch size and Number of >>Threads when this error started coming >> * However when I do touch these parameters (specifically reduce >>them) the error does not come however indexing time increases a lot. >> 4. We have to index overnight daily because we put product prices in >>the Index which get updated nightly >> 5. Solr master is running with a 20 GB Heap >> >>What we have tried >> >> 1. I disabled autoCommit (I.e. Hard commit) and put the autoSoftCommit >>as 5 mins >> * I realized afterwards that this was a wrong test because my >>understanding of soft commit was incorrect, My understanding now is that >>hard commit just truncate the Tlog do hardCommit should be better >>indexing performance. >> * This test failed for lack of space reason however because >>disable autoCommit did not make sense I did not retry this test yet. >> 2. Increased the RAMBufferSizeMB from 100MB to 1000MB >> * This test did not yield anything favorable the master gave the >>early EOF exception >> 3. Increased the merge factor from 20 ‹> 100 >> * This test did not yield anything favorable the master gave the >>early EOF exception >> 4. Flipped the autoCommit to 15 secs and disabled auto commit >> * This test did not yield anything favorable the master gave the >>early EOF exception >> * I got the input for this from >>https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-soft >>commit-and-commit-in-sorlcloud/ - Heavy (Bulk) Indexing section >> 5. Tried to bypass transaction log all together This test is >>underway currently >> >>Questions >> >> 1. Since we are not using solrCloud I want to understand the impact >>of bypassing transaction log >> 2. How does solr take documents which are sent to it to storage as in >>what is the journey of a document from segment to tlog to storage. >> >>It would be great If there are any pointers which you can share. >> >>Thanks >>J./ >> >>The actual Error Log >>ERROR - 2016-08-16 22:59:55.988; org.apache.solr.common.SolrException; >>org.apache.solr.common.SolrException: early EOF >> at >>org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176) >> at >>org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.j >>ava:92) >> at >>org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Content >>StreamHandlerBase.java:74) >> at >>org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas >>e.java:135) >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) >> at >>org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java >>:721) >> at >>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav >>a:417) >> at >>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav >>a:201) >> at >>org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl >>er.java:1419) >> at >>org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) >> at >>org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1 >>37) >> at >>org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557 >>) >> at >>org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.ja >>va:231) >> at >>org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.ja >>va:1075) >> at >>org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) >> at >>org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.jav >>a:193) >> at >>org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.jav >>a:1009) >> at >>org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1 >>35) >> at >>org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHa >>ndlerCollection.java:255) >> at >>org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollectio >>n.java:154) >> at >>org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java >>:116) >> at org.eclipse.jetty.server.Server.handle(Server.java:368) >> at >>org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttp >>Connection.java:489) >> at >>org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttp >>Connection.java:53) >> at >>org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnec >>tion.java:953) >> at >>org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(Abs >>tractHttpConnection.java:1014) >> at >>org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953) >> at >>org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) >> at >>org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnect >>ion.java:72) >> at >>org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketC >>onnector.java:264) >> at >>org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.jav >>a:608) >> at >>org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java >>:543) >> at java.lang.Thread.run(Thread.java:745) >>Caused by: com.ctc.wstx.exc.WstxIOException: early EOF >> at >>com.ctc.wstx.sr.StreamScanner.throwFromIOE(StreamScanner.java:708) >> at >>com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1086) >> at >>org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:389) >> at >>org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246) >> at >>org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174) >> ... 32 more >>Caused by: org.eclipse.jetty.io.EofException: early EOF >> at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65) >> at java.io.InputStream.read(InputStream.java:101) >> at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:365) >> at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:110) >> at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101) >> at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84) >> at >>com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java: >>57) >> at >>com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1046) >> at >>com.ctc.wstx.sr.StreamScanner.parseLocalName2(StreamScanner.java:1796) >> at >>com.ctc.wstx.sr.StreamScanner.parseLocalName(StreamScanner.java:1756) >> at >>com.ctc.wstx.sr.BasicStreamReader.handleNsAttrs(BasicStreamReader.java:298 >>1) >> at >>com.ctc.wstx.sr.BasicStreamReader.handleStartElem(BasicStreamReader.java:2 >>936) >> at >>com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2848 >>) >> at >>com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019) >> ... 35 more >> >
