Thanks shawn for the reply. I would upgrade to solr 4.3 and check that.
On Wed, Jul 31, 2013 at 4:13 PM, Shawn Heisey <s...@elyograg.org> wrote: > On 7/31/2013 4:27 AM, Sinduja Rajendran wrote: > > I am running solr 4.0 in a cloud. We have close to 100Mdocuments. The > data > > is from a single DB table. I use dih. > > Our solrCloud has 3 zookeepers, one tomcat, 2 solr instances in same > > tomcat. We have 8 GB Ram. > > > > After indexing 14M, my indexing fails witht the below exception. > > > > solr org.apache.lucene.index.MergePolicy$MergeException: > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > I tried increasing the GC value to the App server > > > > -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80 > > > > But after giving the command, my indexing went drastically down. Its > > was indexing only 15k documents for 20 minutes. Earlier it was 300k > > for 20 min. > > First thing to mention is that Solr 4.0 was extremely buggy, upgrading > would be advisable. In the meantime: > > An OutOfMemoryError means that Solr needs more heap memory than the JVM > is allowed to use. The Solr Admin UI dashboard will tell you how much > memory is allocated to your JVM, which you can increase with the -Xmx > parameter. Real RAM must be available from the system in order to > increase the heap size. > > The options you have given just change the GC collector and tune one > aspect of the new collector, they don't increase anything. Here are > some things that may help you: > > http://wiki.apache.org/solr/SolrPerformanceProblems > http://wiki.apache.org/solr/ShawnHeisey#GC_Tuning > > After looking over that information and making adjustments, if you are > still having trouble, we can go over your config and all your details to > see what can be done. > > You said that both of your Solr instances are running in the same > tomcat. Just FYI - because you aren't running all functions on separate > hardware, your setup is not fault tolerant. Machine failures DO happen, > no matter how much redundancy you build into that server. If you are > running all this on a redundant VM solution that has live migration of > running VMs, then my statement isn't accurate. > > Thanks, > Shawn > >
