Your channel is full because HDFS Sink is not clearing the channel fast enough.
Thanks, Hari -- Hari Shreedharan On Tuesday, September 18, 2012 at 2:15 PM, Mohit Anchlia wrote: > I have a 6 node cluster and client machines where we run flume. We use Netty > RPC client that comes with flume-ng. When we shutdown one of the nodes in > Hadoop cluster we start get this error and it never stops until the node > comes back up. Could someone suggest what might be wrong here? > > 12/09/17 14:19:48 WARN source.AvroSource: Avro source avroSrc: Unable to > process event. Exception follows. > org.apache.flume.ChannelException: Unable to put event on required channel: > org.apache.flume.channel.MemoryChannel@49938039 > (mailto:org.apache.flume.channel.MemoryChannel@49938039) > at > org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:275) > at org.apache.flume.source.AvroSource.append(AvroSource.java:207) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88) > at org.apache.avro.ipc.Responder.respond(Responder.java:149) > at > org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:178) > at > org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75) > at > org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:163) > at > org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > at > org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792) > at > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) > at > org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:321) > at > org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:303) > at > org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208) > at > org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75) > at > org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > at > org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) > at > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) > at > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) > at > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238) > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38) > at > org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: org.apache.flume.ChannelException: Space for commit to queue > couldn't be acquired Sinks are likely not keeping up with sources, or the > buffer size is too tight > at > org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:91) > at > org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151) > at > org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:267) > ... 28 more
