It might be caused by : 1. Less heap memory. Increasing '-Xms -Xmx' options 2. Disable that prompt. It's not recommended. Using '-XX:-UseGCOverheadLimit' option 3. There always have too many GC limitation with memory channel. I have met some case with you before. I have tried to use direct memory for storing event body out of heap in the past, that's effective method and no any GC happened. I think you can test it by yourself.
-Regards Denny Ye 2012/10/11 Brock Noland <[email protected]> > Yep, sounds like: agent heap size < (capacity * avg event size) > > Brock > > On Wed, Oct 10, 2012 at 8:15 PM, Harish Mandala <[email protected]> > wrote: > > Hi, > > > > In flume-env.sh, please add > > > > JAVA_OPTS="-Xms128m -Xmx256m" > > > > (Or whatever amount of memory works for you. I have Xmx set to 4g) > > > > Regards, > > Harish > > > > > > On Wed, Oct 10, 2012 at 9:08 PM, Camp, Roy <[email protected]> wrote: > >> > >> I ran into the following error – I had to restart flume to recover. Do > I > >> just need to adjust a GC setting of some sort or is there a larger issue > >> here? > >> > >> > >> > >> Source: thriftLegacy > >> > >> Channel: memory (capacity: 1,000,000) > >> > >> Sink: avro > >> > >> > >> > >> > >> > >> Exception in thread "pool-4-thread-2" java.lang.OutOfMemoryError: GC > >> overhead limit exceeded > >> > >> at > >> > java.util.concurrent.LinkedBlockingDeque.<init>(LinkedBlockingDeque.java:108) > >> > >> at > >> > org.apache.flume.channel.MemoryChannel$MemoryTransaction.<init>(MemoryChannel.java:49) > >> > >> at > >> > org.apache.flume.channel.MemoryChannel.createTransaction(MemoryChannel.java:264) > >> > >> at > >> > org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:118) > >> > >> at > >> > org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:260) > >> > >> at > >> > org.apache.flume.source.thriftLegacy.ThriftLegacySource$ThriftFlumeEventServerImpl.append(ThriftLegacySource.java:96) > >> > >> at > >> > com.cloudera.flume.handlers.thrift.ThriftFlumeEventServer$Processor$append.process(ThriftFlumeEventServer.java:276) > >> > >> at > >> > com.cloudera.flume.handlers.thrift.ThriftFlumeEventServer$Processor.process(ThriftFlumeEventServer.java:256) > >> > >> at > >> > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) > >> > >> at > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > >> > >> at > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >> > >> at java.lang.Thread.run(Thread.java:679) > >> > >> Exception in thread > "SinkRunner-PollingRunner-LoadBalancingSinkProcessor" > >> java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> Exception in thread "Avro NettyTransceiver I/O Worker 1" > >> java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> Oct 10, 2012 5:51:39 PM > >> org.jboss.netty.channel.socket.nio.AbstractNioWorker > >> > >> WARNING: Unexpected exception in the selector loop. > >> > >> java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> Oct 10, 2012 5:52:35 PM > >> org.jboss.netty.channel.socket.nio.AbstractNioWorker > >> > >> WARNING: Unexpected exception in the selector loop. > >> > >> java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-2" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "pool-4-thread-4" java.lang.OutOfMemoryError: GC > >> overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-0" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-10" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Oct 10, 2012 5:52:39 PM > >> org.jboss.netty.channel.socket.nio.AbstractNioWorker > >> > >> WARNING: Unexpected exception in the selector loop. > >> > >> java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> Oct 10, 2012 5:52:39 PM > >> org.jboss.netty.channel.socket.nio.AbstractNioWorker > >> > >> WARNING: Unexpected exception in the selector loop. > >> > >> java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-9" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-1" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-5" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-3" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-13" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-6" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-4" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-5" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Exception in thread "lifecycleSupervisor-1-8" > java.lang.OutOfMemoryError: > >> GC overhead limit exceeded > >> > >> Oct 10, 2012 5:52:48 PM > >> org.jboss.netty.channel.socket.nio.AbstractNioWorker > >> > >> WARNING: Unexpected exception in the selector loop. > >> > >> > >> > >> > >> > >> Thanks, > >> > >> > >> > >> Roy > >> > >> > >> > >> > > > > > > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ >
