We're now running several solr instances on quad-cores and getting fairly good
RPS even on
the largest index (26MM documents) after implementing faceted queries. Things
are looking good
except for this OutOfMemoryError which occurs every 2 hrs at peak.
Note: I have browsed, searched the forums for this error and followed the most
common advice of
increasing the memory allocation for the JVM:
/usr/bin/java -DSTOP.PORT=8805 -DSTOP.KEY=solrstop -Xmx3584M -Xms1024M -jar
start.jar
I have also reduced the autowarmcounts to 100 from 2000. Still, after running a
couple of hrs or
so solr/ping returns 500 error with (see below) the error in the log. We're
running on jetty and
the index size is 3.7Gb with 26MM documents. I don't see this error in other
solr instances which
have an index of 1.3Gb and 8MM docs (those simply hang, which is another topic,
but it's more rare)
I could allocate more physical memory, but I can't seem to increase the -Xmx
option to 3800 I get
an error : "Could not reserve enough space for object heap", even though I have
more than 4Gb free.
(We're running on Intel quad core 64bit) When I try strace I'm seeing mmap2
errors.
So, question which comes to mind - can this problem be solved otherwise? I'm
sure others have used
larger indexes than this. What other settings can I tweak to get rid of this
error... I'm currently running
a healthcheck and simply restart the solr instance when I get a 500 error on
the solr/ping. But this is
ugly and bad for cache... Any ideas?
Thanks in advance!
-Alex
Jan 24, 2008 3:25:44 PM org.apache.solr.core.SolrException log
SEVERE: java.lang.OutOfMemoryError: Java heap space
at org.apache.solr.util.OpenBitSet.clone(OpenBitSet.java:539)
at org.apache.solr.search.DocSetBase.intersection(DocSet.java:201)
at
org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:568)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:788)
at
org.apache.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.java:698)
at
org.apache.solr.request.StandardRequestHandler.handleRequestBody(StandardRequestHandler.java:122)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:658)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
