Which machine is giving you the errors? host1 or host2? You are probably exceeding the capacity of the channel, as the exception suggests. Have you tried increasing the channel capacity? Looks like you are using the default queue size of 100. Try something more like this:
host1.channels.memoryChannel.capacity = 100000 As Alex says, it's also a good idea to set JAVA_OPTS="-Xmx512m" or something like that in your flume-env.sh to bump up the memory size if you run into OutOfMemoryErrors. Depends on the size of your events and how bursty you are, as well as the speed of your sink vs. the speed of your source. HTH, Mike On Apr 26, 2012, at 1:03 AM, shekhar sharma wrote: > Hello Alex, > My configuration file is as follows: > > host1.properties file > > host1.sources = avroSource > host1.channels = memoryChannel > host1.sinks = esper > > #avroSource configuration > > host1.sources.avroSource.type = avro > host1.sources.avroSource.bind = localhost > host1.sources.avroSource.port = 41414 > host1.sources.avroSource.channels = memoryChannel > > #Channels > > host1.channels.memoryChannel.type = memory > > > #Sinks > host1.sinks.esper.type = org.apache.flume.sink.esper.EsperSink > host1.sinks.esper.channel = memoryChannel > > host2.properties file: > > host2.sources = execSource > host2.channels = memoryChannel > host2.sinks = avroSink > > #execSource configuration > > host2.sources.execSource.type = exec > host2.sources.execSource.command = /usr/bin/tail -F > /home/dev/LogScripts/json.csv > host2.sources.execSource.channels = memoryChannel > > #Channels > > host2.channels.memoryChannel.type = memory > > > #Sinks > host2.sinks.avroSink.type = avro > host2.sinks.avroSink.hostname=localhost > host2.sinks.avroSink.port=41414 > host2.sinks.avroSink.batch-size = 10 > host2.sinks.avroSink.runner.type = polling > host2.sinks.avroSink.runner.polling.interval = 1 > host2.sinks.avroSink.channel = memoryChannel > > Regards, > Som Shekhar Sharma > > > On Thu, Apr 26, 2012 at 12:35 PM, shekhar sharma <shekhar2...@gmail.com> > wrote: > Hello, > While using memory channel i am getting the following error, what could be > the reason for this: > org.apache.flume.ChannelException: Space for commit to queue couldn't be > acquired Sinks are likely not keeping up with sources, or the buffer size is > too tight > > > org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:84) > at > org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151) > at > org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:178) > at > org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:267) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > > > Regards, > Som >
