RE: entire farm fails at the same time with OOM issues

2010-12-09 Thread Chris Hostetter
I'm not sure if you resolved this issue, but... : It has typically been when query traffic was lowest! We are at 12 GB ...that doesn't mean it couldn't have been query load related. it's possible that some unusual query (ie: trying to sort on many fields at the same time?) could have forced

RE: entire farm fails at the same time with OOM issues

2010-12-01 Thread Robert Petersen
Good idea. Our farm is behind Akamai so that should be ok to do. -Original Message- From: Peter Karich [mailto:peat...@yahoo.de] Sent: Wednesday, December 01, 2010 12:21 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues also try to

Re: entire farm fails at the same time with OOM issues

2010-12-01 Thread Peter Karich
also try to minimize maxWarming searchers to 1(?) or 2. And decrease cache usage (especially autowarming) if possible at all. But again: only if it doesn't affect performance ... Regards, Peter. On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersen wrote: My question is this. Why in the world

Re: entire farm fails at the same time with OOM issues

2010-12-01 Thread Ken Krugler
10 3:12 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues Hi Robert, I'd recommend launching Tomcat with -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=, so then you have something to look at versus a Gedankenexperiment :) -- Ken On Nov

RE: entire farm fails at the same time with OOM issues

2010-12-01 Thread Robert Petersen
lMode" -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Tuesday, November 30, 2010 6:44 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersen w

Re: entire farm fails at the same time with OOM issues

2010-11-30 Thread Yonik Seeley
On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersen wrote: > My question is this.  Why in the world would all of my slaves, after > running fine for some days, suddenly all at the exact same minute > experience OOM heap errors and go dead? If there is no change in query traffic when this happens, th

RE: entire farm fails at the same time with OOM issues

2010-11-30 Thread Robert Petersen
add up to anything right? -Original Message- From: Ken Krugler [mailto:kkrugler_li...@transpac.com] Sent: Tuesday, November 30, 2010 3:12 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues Hi Robert, I'd recommend launching Tomcat wit

Re: entire farm fails at the same time with OOM issues

2010-11-30 Thread Ken Krugler
Hi Robert, I'd recommend launching Tomcat with -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=, so then you have something to look at versus a Gedankenexperiment :) -- Ken On Nov 30, 2010, at 3:04pm, Robert Petersen wrote: Greetings, we are running one master and four slaves of our

entire farm fails at the same time with OOM issues

2010-11-30 Thread Robert Petersen
Greetings, we are running one master and four slaves of our multicore solr setup. We just served searches for our catalog of 8 million products with this farm during black Friday and cyber Monday, our busiest days of the year, and the servers did not break a sweat! Index size is about 28GB. H