HI Alex, Everything is working fine, Esper is able to detect the events and can write them to separate log file. Well i have written a python script which generates the log file and i am creating almost 60000 records, and if you can see that my exec source tail command is working on that log file... After the events are detected and processed by Esper, and this error does not comes instantenously, after some time it is thrown...
Could it me because of -Xmx20 options ?..because the system is my virtual linux box with 1GB RAM and 20GB HD.. Can the size of the event can also effect? the reason why i am saying this is: Event read by exec source is of SimpleEvent which has 2 member variable MAP for headers and byte[] for body and in Esper if you want to employ filter "IN" or "LIKE" then it requires string..so what i did, i have created another member variable of type string to convert the byte array to string. SO while sending this event to the channel, is this bloating up the channel capacity? Regards, Som On Thu, Apr 26, 2012 at 1:41 PM, alo alt <wget.n...@googlemail.com> wrote: > Thanks, > > org.apache.flume.sink.esper.EsperSink is availabe in the CLASSPATH and > flume can load them? Or did you write the sink? > > best, > Alex > > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Apr 26, 2012, at 10:03 AM, shekhar sharma wrote: > > > Hello Alex, > > My configuration file is as follows: > > > > host1.properties file > > > > host1.sources = avroSource > > host1.channels = memoryChannel > > host1.sinks = esper > > > > #avroSource configuration > > > > host1.sources.avroSource.type = avro > > host1.sources.avroSource.bind = localhost > > host1.sources.avroSource.port = 41414 > > host1.sources.avroSource.channels = memoryChannel > > > > #Channels > > > > host1.channels.memoryChannel.type = memory > > > > > > #Sinks > > host1.sinks.esper.type = org.apache.flume.sink.esper.EsperSink > > host1.sinks.esper.channel = memoryChannel > > > > host2.properties file: > > > > host2.sources = execSource > > host2.channels = memoryChannel > > host2.sinks = avroSink > > > > #execSource configuration > > > > host2.sources.execSource.type = exec > > host2.sources.execSource.command = /usr/bin/tail -F > /home/dev/LogScripts/json.csv > > host2.sources.execSource.channels = memoryChannel > > > > #Channels > > > > host2.channels.memoryChannel.type = memory > > > > > > #Sinks > > host2.sinks.avroSink.type = avro > > host2.sinks.avroSink.hostname=localhost > > host2.sinks.avroSink.port=41414 > > host2.sinks.avroSink.batch-size = 10 > > host2.sinks.avroSink.runner.type = polling > > host2.sinks.avroSink.runner.polling.interval = 1 > > host2.sinks.avroSink.channel = memoryChannel > > > > Regards, > > Som Shekhar Sharma > > > > > > On Thu, Apr 26, 2012 at 12:35 PM, shekhar sharma <shekhar2...@gmail.com> > wrote: > > Hello, > > While using memory channel i am getting the following error, what could > be the reason for this: > > org.apache.flume.ChannelException: Space for commit to queue couldn't be > acquired Sinks are likely not keeping up with sources, or the buffer size > is too tight > > > > > > > org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:84) > > at > org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151) > > at > org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:178) > > at > org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:267) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > > > > > Regards, > > Som > > > >
