GC overhead exceeded means more than 99% CPU spent of GC. Reason is you are creating too many/ too complex objects.
Any chance you can reduce amount of object you create? Allocation view of a profiler can help. --Srinath On Mon, Mar 3, 2014 at 10:51 PM, Chamila Wijayarathna <[email protected]>wrote: > Hi, > I got a similar error when running a cassandra archive job on BAM. By > analyzing "heap-dump.hprof" file, using Eclipse Memory Analyzer, I found > that 1 memory table has grow more than 600MB. > In this case, what I did as solution is reducing > "flush_largest_memtables_at" value in cassandra.yaml from 0.75 to 0.4. > As Dr. Srinath explained, jvm starts to flush its largest memory table at > the point it jvm consume 0.75 of memory. But before that process finishes > memory overflow can occur. But reducing that value this can be solved if > this occurs due to same reason. I am not sure if this works in ESB case. > > Thank You! > > > On Mon, Mar 3, 2014 at 10:09 PM, Iranga Muthuthanthri <[email protected]>wrote: > >> Thanks Udara. Checked it before and the stack trace was different.The >> culprit in this case appears to from the andes client. >> On Mar 3, 2014 9:25 PM, "Udara Liyanage" <[email protected]> wrote: >> >>> I saw a similar issue occurred in BAM. Have a look mail thread subjected >>> " [Dev] [BAM] java.lang.OutOfMemoryError: GC overhead limit exceeded" >>> >>> Touched, not typed. Erroneous words are a feature, not a typo. >>> On Mar 3, 2014 6:57 PM, "Iranga Muthuthanthri" <[email protected]> wrote: >>> >>>> Hi All. >>>> >>>> Following was noted in testing for 100,00 messages for the ESB (4.8.1) >>>> message store and forward scenario in a MB 2.1.0 cluster environment. >>>> >>>> *Issue*:Out of memory exception noted in ESB console. >>>> >>>> [2014-03-03 04:37:36,065] ERROR - JmsStore Could not create a Message >>>> Producer for Store [MessageStoreO1]. Error:GC overhead limit exceeded >>>> java.lang.OutOfMemoryError: GC overhead limit exceeded >>>> at java.util.Arrays.copyOfRange(Arrays.java:3209) >>>> at java.lang.String.<init>(String.java:215) >>>> at java.lang.StringBuilder.toString(StringBuilder.java:430) >>>> at java.lang.Object.toString(Object.java:219) >>>> at >>>> org.wso2.andes.client.BasicMessageProducer.setPublishMode(BasicMessageProducer.java:167) >>>> at >>>> org.wso2.andes.client.BasicMessageProducer.<init>(BasicMessageProducer.java:148) >>>> at >>>> org.wso2.andes.client.BasicMessageProducer_0_8.<init>(BasicMessageProducer_0_8.java:51) >>>> at >>>> org.wso2.andes.client.AMQSession_0_8.createMessageProducer(AMQSession_0_8.java:411) >>>> at >>>> org.wso2.andes.client.AMQSession_0_8.createMessageProducer(AMQSession_0_8.java:82) >>>> at org.wso2.andes.client.AMQSession$7.execute(AMQSession.java:2632) >>>> at org.wso2.andes.client.AMQSession$7.execute(AMQSession.java:2627) >>>> at >>>> org.wso2.andes.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:315) >>>> at >>>> org.wso2.andes.client.AMQConnection.executeRetrySupport(AMQConnection.java:621) >>>> at >>>> org.wso2.andes.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102) >>>> at >>>> org.wso2.andes.client.AMQSession.createProducerImpl(AMQSession.java:2625) >>>> at >>>> org.wso2.andes.client.AMQSession.createProducerImpl(AMQSession.java:2619) >>>> at >>>> org.wso2.andes.client.AMQSession.createProducer(AMQSession.java:1231) >>>> at org.wso2.andes.client.AMQSession.createProducer(AMQSession.java:109) >>>> at >>>> org.apache.synapse.message.store.impl.jms.JmsStore.newProducer(JmsStore.java:346) >>>> at >>>> org.apache.synapse.message.store.impl.jms.JmsStore.getProducer(JmsStore.java:136) >>>> at >>>> org.apache.synapse.mediators.store.MessageStoreMediator.mediate(MessageStoreMediator.java:84) >>>> at >>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:77) >>>> at >>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:47) >>>> at >>>> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:131) >>>> at >>>> org.apache.synapse.core.axis2.ProxyServiceMessageReceiver.receive(ProxyServiceMessageReceiver.java:166) >>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>> at >>>> org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:411) >>>> at >>>> org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:183) >>>> at >>>> org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>> at java.lang.Thread.run(Thread.java:662) >>>> >>>> >>>> Issue details can be found on:https://wso2.org/jira/browse/ESBJAVA-3024 >>>> >>>> >>>> >>>> -- >>>> Thanks & Regards >>>> >>>> Iranga Muthuthanthri >>>> Technical Lead - QA >>>> (M) -0777-255773 >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > *Chamila Wijayarathna* > Engineering Intern, > WSO2 Inc. > > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- ============================ Srinath Perera, Ph.D. Director, Research, WSO2 Inc. Visiting Faculty, University of Moratuwa Member, Apache Software Foundation Research Scientist, Lanka Software Foundation Blog: http://srinathsview.blogspot.com/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
