Hello Alexander, I have created a class known as EsperSink.java and EsperListener.java, compiled them and created a jar(esper-sink.jar) file.
I have put the esper-sink.jar along with all esper related jar into the lib folder (/flume-1.1.0-incubating-SNAPSHOT/lib) And my agent file is something like this: # Sources agent1.sources.log.type = exec agent1.sources.log.command=/usr/bin/tail -F /home/dev/EsperPOC/SamplePOC/test.txt agent1.sources.log.channels = log # Channgels agent1.channels.log.type = memory # Sinks agent1.sinks.log.type = Esper agent1.sinks.log.hostname = namenode agent1.sinks.log.port = 41414 agent1.sinks.log.batch-size = 10 agent1.sinks.log.runner.type = polling agent1.sinks.log.runner.polling.interval = 5 agent1.sinks.log.channel = log # Load everything agent1.sources = log agent1.sinks = log agent1.channels = log When i am running the agent, it is throwing the following error: 2012-03-27 20:50:50,317 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows. org.apache.flume.FlumeException: Unable to load sink type: Esper, class: Esper There is no file with a name flume-conf.xml only property template and this property template is meant for defining sources and sinks. Anyways i will create flume-conf.xml and will add the plugins Regards, Som On Wed, Mar 28, 2012 at 6:59 PM, alo alt <[email protected]> wrote: > You have to put the them into ./lib (from a flume view) and enable the > jars in flume-conf.xml: > > <property> > <name>flume.plugin.classes</name> > <value>here.the.plugins</value> > <description>Comma separated list of plugin classes</description> > </property> > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Mar 28, 2012, at 12:15 PM, shekhar sharma wrote: > > > Hel, > > i have written a custom sink for Esper, can you please how would i make > FLume to recognize this is my new sink..Do i have to specify in > configuration file. Can you please help me? > > > > Regards, > > Som > > > > On Wed, Mar 28, 2012 at 1:56 PM, shekhar sharma <[email protected]> > wrote: > > Thanks for a quick response.. > > But the sinks written for cloudera flume (i was going through hello > world plugin) use cloudera classes and all.. For FLume-728 the sinks > written (HDFSSink and IRCSink) is different from cloudera's. > > CLoudera flume requires cloudera's hadoop to be installed, if i am not > wrong? > > > > My idea is to collect the events from Flume agent and send to Esper, > where i will query the streams using EPL. > > Can you please tell me what all things are required to write a custom > sink. i mean what needs to be done so that Flume node recognizes the sink. > > > > Thanks > > > > Regards, > > Som > > > > > > On Wed, Mar 28, 2012 at 1:06 PM, alo alt <[email protected]> > wrote: > > Hi, > > > > Cloudera uses in 3u3 Flume-798 > > > > Apache flume isn't different, you can build it among your installation > and tweak the configs to match them or use the binary distribution: > > https://github.com/cloudera/flume/downloads > > > > As you read, apache flume 1.1.0 was just released: > > http://www.apache.org/dyn/closer.cgi/incubator/flume/ > > > > You have to write plugins for sinks, here you found some created from > the community: > > https://cwiki.apache.org/FLUME/flume-plugins.html > > > > And the docs: > > > http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins > > > > Q3 - I don't know, if you use serialized data whats with avro? > > > > - Alex > > > > -- > > Alexander Lorenz > > http://mapredit.blogspot.com > > > > On Mar 28, 2012, at 5:49 AM, shekhar sharma wrote: > > > > > Hello every one, > > > I am trying to integrate FLume and Esper by writing a custom sink for > Esper. But as per the documentation , i am unable to do so. i have tried > the hello world plugin, but that one also not working. > > > I am using FLume-728. > > > I have few question: > > > > > > Q1. Is Apache Flume is different from Cloudera's Flume? > > > If i am using Cloudera's FLume, then does it have any > dependency that i have to use CLoudera's distribution of Hadoop. > > > > > > Q2. What are the necessary steps for writing custom sink ? > > > > > > > > > Q3. I have implemented a POC in Esper, and it takes POJO (Plain old > java objects), XML or Map as events. Now if i try integrating both, how > feasible it is? > > > > > > Regards, > > > Som Shekhar Sharma > > > > > > > > > > > > > >
