Hi, In flume-env.sh, please add
JAVA_OPTS="-Xms128m -Xmx256m" (Or whatever amount of memory works for you. I have Xmx set to 4g) Regards, Harish On Wed, Oct 10, 2012 at 9:08 PM, Camp, Roy <[email protected]> wrote: > I ran into the following error – I had to restart flume to recover. Do > I just need to adjust a GC setting of some sort or is there a larger issue > here?**** > > ** ** > > Source: thriftLegacy**** > > Channel: memory (capacity: 1,000,000)**** > > Sink: avro**** > > ** ** > > ** ** > > Exception in thread "pool-4-thread-2" java.lang.OutOfMemoryError: GC > overhead limit exceeded**** > > at > java.util.concurrent.LinkedBlockingDeque.<init>(LinkedBlockingDeque.java:108) > **** > > at > org.apache.flume.channel.MemoryChannel$MemoryTransaction.<init>(MemoryChannel.java:49) > **** > > at > org.apache.flume.channel.MemoryChannel.createTransaction(MemoryChannel.java:264) > **** > > at > org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:118) > **** > > at > org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:260) > **** > > at > org.apache.flume.source.thriftLegacy.ThriftLegacySource$ThriftFlumeEventServerImpl.append(ThriftLegacySource.java:96) > **** > > at > com.cloudera.flume.handlers.thrift.ThriftFlumeEventServer$Processor$append.process(ThriftFlumeEventServer.java:276) > **** > > at > com.cloudera.flume.handlers.thrift.ThriftFlumeEventServer$Processor.process(ThriftFlumeEventServer.java:256) > **** > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) > **** > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > **** > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > **** > > at java.lang.Thread.run(Thread.java:679)**** > > Exception in thread "SinkRunner-PollingRunner-LoadBalancingSinkProcessor" > java.lang.OutOfMemoryError: GC overhead limit exceeded**** > > Exception in thread "Avro NettyTransceiver I/O Worker 1" > java.lang.OutOfMemoryError: GC overhead limit exceeded**** > > Oct 10, 2012 5:51:39 PM > org.jboss.netty.channel.socket.nio.AbstractNioWorker**** > > WARNING: Unexpected exception in the selector loop.**** > > java.lang.OutOfMemoryError: GC overhead limit exceeded**** > > Oct 10, 2012 5:52:35 PM > org.jboss.netty.channel.socket.nio.AbstractNioWorker**** > > WARNING: Unexpected exception in the selector loop.**** > > java.lang.OutOfMemoryError: GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-2" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "pool-4-thread-4" java.lang.OutOfMemoryError: GC > overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-0" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-10" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Oct 10, 2012 5:52:39 PM > org.jboss.netty.channel.socket.nio.AbstractNioWorker**** > > WARNING: Unexpected exception in the selector loop.**** > > java.lang.OutOfMemoryError: GC overhead limit exceeded**** > > Oct 10, 2012 5:52:39 PM > org.jboss.netty.channel.socket.nio.AbstractNioWorker**** > > WARNING: Unexpected exception in the selector loop.**** > > java.lang.OutOfMemoryError: GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-9" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-1" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-5" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-3" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-13" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-6" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-4" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-5" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Exception in thread "lifecycleSupervisor-1-8" java.lang.OutOfMemoryError: > GC overhead limit exceeded**** > > Oct 10, 2012 5:52:48 PM > org.jboss.netty.channel.socket.nio.AbstractNioWorker**** > > WARNING: Unexpected exception in the selector loop.**** > > ** ** > > ** ** > > Thanks,**** > > ** ** > > Roy**** > > ** ** > > ** ** >
