Hello Alex, One more bit of information is that out of 60000 records, previously i could process all the 60000 records, but not Esper is logging only 5000 records..
Regards, Som On Thu, Apr 26, 2012 at 1:54 PM, shekhar sharma <shekhar2...@gmail.com>wrote: > HI Alex > Everything is working fine, Esper is able to detect the events and can > write them to separate log file. > Well i have written a python script which generates the log file and i am > creating almost 60000 records, and if you can see that my exec source tail > command is working on that log file... > After the events are detected and processed by Esper, and this error does > not comes instantenously, after some time it is thrown... > > Could it me because of -Xmx20 options ?..because the system is my virtual > linux box with 1GB RAM and 20GB HD.. > Can the size of the event can also effect? the reason why i am saying this > is: > > Event read by exec source is of SimpleEvent which has 2 member variable > MAP for headers and byte[] for body and in Esper if you want to employ > filter "IN" or "LIKE" then it requires string..so what i did, i have > created another member variable of type string to convert the byte array to > string. SO while sending this event to the channel, is this bloating up the > channel capacity? > > Regards, > Som > > > > On Thu, Apr 26, 2012 at 1:41 PM, alo alt <wget.n...@googlemail.com> wrote: > >> Thanks, >> >> org.apache.flume.sink.esper.EsperSink is availabe in the CLASSPATH and >> flume can load them? Or did you write the sink? >> >> best, >> Alex >> >> >> -- >> Alexander Lorenz >> http://mapredit.blogspot.com >> >> On Apr 26, 2012, at 10:03 AM, shekhar sharma wrote: >> >> > Hello Alex, >> > My configuration file is as follows: >> > >> > host1.properties file >> > >> > host1.sources = avroSource >> > host1.channels = memoryChannel >> > host1.sinks = esper >> > >> > #avroSource configuration >> > >> > host1.sources.avroSource.type = avro >> > host1.sources.avroSource.bind = localhost >> > host1.sources.avroSource.port = 41414 >> > host1.sources.avroSource.channels = memoryChannel >> > >> > #Channels >> > >> > host1.channels.memoryChannel.type = memory >> > >> > >> > #Sinks >> > host1.sinks.esper.type = org.apache.flume.sink.esper.EsperSink >> > host1.sinks.esper.channel = memoryChannel >> > >> > host2.properties file: >> > >> > host2.sources = execSource >> > host2.channels = memoryChannel >> > host2.sinks = avroSink >> > >> > #execSource configuration >> > >> > host2.sources.execSource.type = exec >> > host2.sources.execSource.command = /usr/bin/tail -F >> /home/dev/LogScripts/json.csv >> > host2.sources.execSource.channels = memoryChannel >> > >> > #Channels >> > >> > host2.channels.memoryChannel.type = memory >> > >> > >> > #Sinks >> > host2.sinks.avroSink.type = avro >> > host2.sinks.avroSink.hostname=localhost >> > host2.sinks.avroSink.port=41414 >> > host2.sinks.avroSink.batch-size = 10 >> > host2.sinks.avroSink.runner.type = polling >> > host2.sinks.avroSink.runner.polling.interval = 1 >> > host2.sinks.avroSink.channel = memoryChannel >> > >> > Regards, >> > Som Shekhar Sharma >> > >> > >> > On Thu, Apr 26, 2012 at 12:35 PM, shekhar sharma <shekhar2...@gmail.com> >> wrote: >> > Hello, >> > While using memory channel i am getting the following error, what could >> be the reason for this: >> > org.apache.flume.ChannelException: Space for commit to queue couldn't >> be acquired Sinks are likely not keeping up with sources, or the buffer >> size is too tight >> > >> > >> > >> org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:84) >> > at >> org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151) >> > at >> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:178) >> > at >> org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:267) >> > at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >> > at >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >> > at java.util.concurrent.FutureTask.run(FutureTask.java:138) >> > at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> > at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> > at java.lang.Thread.run(Thread.java:662) >> > >> > >> > Regards, >> > Som >> > >> >> >
