[ https://issues.apache.org/jira/browse/SOLR-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar resolved SOLR-1482. ----------------------------------------- Resolution: Cannot Reproduce This is a very old issue. I haven't heard of similar hangs happening recently. Please let us know if anyone thinks it should be re-opened. > Solr master and slave freeze after query > ---------------------------------------- > > Key: SOLR-1482 > URL: https://issues.apache.org/jira/browse/SOLR-1482 > Project: Solr > Issue Type: Bug > Affects Versions: 1.4 > Environment: Nightly 9/28/09. > 14 individual instances per server, using JNDI. > replicateAfter commit, 5 min interval polling. > All caches are currently commented out, on both slave and master. > Lots of ongoing commits - large chunks of data, each accompanied by a commit. > This is to guarantee that anything we think is now in Solr remains there in > case the server crashes. > Reporter: Artem Russakovskii > Priority: Critical > Attachments: catalina.out, catalina2.out > > > We're having issues with the deployment of 2 master-slave setups. > One of the master-slave setups is OK (so far) but on the other both the > master and the slave keep freezing, but only after I send a query to them. > And by freezing I mean indefinite hanging, with almost no output to log, no > errors, nothing. It's as if there's some sort of a deadlock. The hanging > servers need to be killed with -9, otherwise they keep hanging. > The query I send queries all instances at the same time using the ?shards= > syntax. > On the slave, the logs just stop - nothing shows up anymore after the query > is issued. On the master, they're a bit more descriptive. This information > seeps through very-very slowly, as you can see from the timestamps: > {quote} > SEVERE: java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:16:00 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:19:37 PM org.apache.catalina.connector.CoyoteAdapter service > SEVERE: An exception or error occurred in the container during the request > processing > java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:19:37 PM org.apache.coyote.http11.Http11Processor process > SEVERE: Error processing request > java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:19:39 PM org.apache.catalina.connector.CoyoteAdapter service > SEVERE: An exception or error occurred in the container during the request > processing > java.lang.OutOfMemoryError: PermGen space > Exception in thread "ContainerBackException in thread "pool-29-threadOct 1, > 2009 2:21:47 PM org.apache.catalina.connector.CoyoteAdapter service > SEVERE: An exception or error occurred in the container during the request > processing > java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:21:47 PM > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process > SEVERE: Error reading request, ignored > java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:21:47 PM > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process > SEVERE: Error reading request, ignored > java.lang.OutOfMemoryError: PermGen space > -22" java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:21:47 PM > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process > SEVERE: Error reading request, ignored > java.lang.OutOfMemoryError: PermGen space > Exception in thread "http-8080-42" Oct 1, 2009 2:21:47 PM > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process > SEVERE: Error reading request, ignored > java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:21:47 PM > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process > SEVERE: Error reading request, ignored > java.lang.OutOfMemoryError: PermGen space > Oct 1, 2009 2:21:47 PM > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process > SEVERE: Error reading request, ignored > java.lang.OutOfMemoryError: PermGen space > Exception in thread "http-8080-26" Exception in thread "http-8080-32" > Exception in thread "http-8080-25" Exception in thread "http-8080-22" > Exception in thread "http-8080-15" Exception in thread "http-8080-45" > Exception in thread "http-8080-13" Exception in thread "http-8080-48" > Exception in thread "http-8080-7" Exception in thread "http-8080-38" > Exception in thread "http-8080-39" Exception in thread "http-8080-28" > Exception in thread "http-8080-1" Exception in thread "http-8080-2" Exception > in thread "http-8080-12" Exception in thread "http-8080-44" Exception in > thread "http-8080-47" Exception in thread "http-8080-29" Exception in thread > "http-8080-33" Exception in thread "http-8080-27" Exception in thread > "http-8080-36" Exception in thread "http-8080-113" Exception in thread > "http-8080-112" Exception in thread "http-8080-37" Exception in thread > "http-8080-18" java.lang.OutOfMemoryError: PermGen space > java.lang.OutOfMemoryError: PermGen space > java.lang.OutOfMemoryError: PermGen space > java.lang.OutOfMemoryError: PermGen space > java.lang.OutOfMemoryError: PermGen space > Exception in thread "http-8080-34" java.lang.OutOfMemoryError: PermGen space > java.lang.OutOfMemoryError: PermGen space > Exception in thread "http-8080-103" > {quote} > So the problem seems to be related to PermGen space. I found > http://www.nabble.com/Number-of-webapps-td22198080.html and tried > -XX:MaxPermSize=256m, but it didn't fix the problem. The current > CATALINA_OPTS looks like this: > {quote} > export CATALINA_OPTS="-XX:MaxPermSize=256m -Xmx6500m -XX:+UseConcMarkSweepGC" > {quote} > Is the only solution at this point going multicore, as Noble suggested (is > Noble your first name? I always assumed it was Paul and Noble was part of the > nickname)? Will multicore get rid of the problem, before we spend time > looking at it? For multicore, will the existing data dirs be compatible or > would a complete reindex be needed? > I'm willing to provide any information to you guys, just not sure what at the > moment. I'm also open to communicating outside of JIRA, at artem [_aT_] plaxo > {dot} com. > Thanks. -- This message was sent by Atlassian JIRA (v6.1#6144) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org