It looks lik that error happens when reading XML from an HTTP request. The XML ends too soon. This should be unrelated to file buffers.
wunder On Aug 26, 2013, at 9:17 AM, Furkan KAMACI wrote: > It has a 48 GB of RAM and index size is nearly 100 GB at each node. I have > CentOS 6.4. While indexing I got that error and I am suspicious about that > it is because of high percentage of Physical Memory usage. > > ERROR - 2013-08-21 22:01:30.979; org.apache.solr.common.SolrException; > java.lang.RuntimeException: [was class org.eclipse.jetty.io.EofException] > early EOF > at > com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18) > at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731) > at > com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657) > at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809) > at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:393) > at > org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:245) > at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173) > at > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) > at > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1812) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > at org.eclipse.jetty.server.Server.handle(Server.java:365) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) > at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) > at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) > at > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) > at java.lang.Thread.run(Thread.java:722) > Caused by: org.eclipse.jetty.io.EofException: early EOF > at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65) > at java.io.InputStream.read(InputStream.java:101) > at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:365) > at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:110) > at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101) > at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84) > at > com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57) > at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992) > at > com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628) > at > com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126) > at > com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701) > at > com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649) > ... 36 more > > > > 2013/8/26 Walter Underwood <wun...@wunderwood.org> > >> What is the precise error? What kind of machine? >> >> File buffers are a robust part of the OS. Unix has had file buffer caching >> for decades. >> >> wunder >> >> On Aug 26, 2013, at 1:37 AM, Furkan KAMACI wrote: >> >>> Hi Walter; >>> >>> You are right about performance. However when I index documents on a >>> machine that has a high percentage of Physical Memory usage I get EOF >>> errors? >>> >>> >>> 2013/8/26 Walter Underwood <wun...@wunderwood.org> >>> >>>> On Aug 25, 2013, at 1:41 PM, Furkan KAMACI wrote: >>>> >>>>> Sometimes Physical Memory usage of Solr is over %99 and this may cause >>>>> problems. Do you run such kind of a command periodically: >>>>> >>>>> sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches" >>>>> >>>>> to force dropping caches of machine that Solr runs at and avoid >> problems? >>>> >>>> >>>> This is a terrible idea. The OS automatically manages the file buffers. >>>> When they are all used, that is a good thing, because it reduced disk >> IO. >>>> >>>> After this, no files will be cached in RAM. Every single read from a >> file >>>> will have to go to disk. This will cause very slow performance until the >>>> files are recached. >>>> >>>> Recently, I did exactly the opposite to improve performance in our Solr >>>> installation. Before starting the Solr process, a script reads every >> file >>>> in the index so that it will already be in file buffers. This avoids >>>> several minutes of high disk IO and slow performance after startup. >>>> >>>> wunder >>>> Search Guy, Chegg.com >>>> >>>> >>>> >> >> -- >> Walter Underwood >> wun...@wunderwood.org >> >> >> >> -- Walter Underwood wun...@wunderwood.org
