On Thu, Oct 1, 2009 at 11:41 AM, Jeff Newburn <jnewb...@zappos.com> wrote: > I am trying to update to the newest version of solr from trunk as of May > 5th.
Tons of changes since... including the per-segment searching/sorting/function queries (I think). Do you sort on any single valued fields that you also facet on? Do you use ord() or rord() in any function queries? Unfortunately, some of these things will take up more memory because some things still cache FieldCache elements with the top-level reader, while some use segment readers. The direction is going toward all segment readers, but we're not there yet (and won't be for 1.4). ord() rord() will never be fixed... people need to migrate to something else. http://issues.apache.org/jira/browse/SOLR-1111 is the main issue for this. If course, I've really only been talking about search related changes. Nothing on the indexing side should cause greater memory usage.... but perhaps the indexing side could run out of memory due to the search side taking up more. -Yonik http://www.lucidimagination.com > I updated and compiled from trunk as of yesterday (09/30/2009). When > I try to do a full import I am receiving a GC heap error after changing > nothing in the configuration files. Why would this happen in the most > recent versions but not in the version from a few months ago. The stack > trace is below. > > Oct 1, 2009 8:34:32 AM org.apache.solr.update.processor.LogUpdateProcessor > finish > INFO: {add=[166400, 166608, 166698, 166800, 166811, 167097, 167316, 167353, > ...(83 more)]} 0 35991 > Oct 1, 2009 8:34:32 AM org.apache.solr.common.SolrException log > SEVERE: java.lang.OutOfMemoryError: GC overhead limit exceeded > at java.util.Arrays.copyOfRange(Arrays.java:3209) > at java.lang.String.<init>(String.java:215) > at com.ctc.wstx.util.TextBuffer.contentsAsString(TextBuffer.java:384) > at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:821) > at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:280) > at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:139) > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69) > at > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentSt > reamHandlerBase.java:54) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. > java:131) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3 > 38) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: > 241) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application > FilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh > ain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja > va:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja > va:175) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128 > ) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102 > ) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java > :109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) > at > org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java: > 879) > at > org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(H > ttp11NioProtocol.java:719) > at > org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java: > 2080) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja > va:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 > 08) > at java.lang.Thread.run(Thread.java:619) > > Oct 1, 2009 8:40:06 AM org.apache.solr.core.SolrCore execute > INFO: [zeta-main] webapp=/solr path=/update params={} status=500 QTime=5265 > Oct 1, 2009 8:40:12 AM org.apache.solr.common.SolrException log > SEVERE: java.lang.OutOfMemoryError: GC overhead limit exceeded > > -- > Jeff Newburn > Software Engineer, Zappos.com > jnewb...@zappos.com - 702-943-7562 > >
