Hel, i have written a custom sink for Esper, can you please how would i make FLume to recognize this is my new sink..Do i have to specify in configuration file. Can you please help me?
Regards, Som On Wed, Mar 28, 2012 at 1:56 PM, shekhar sharma <[email protected]>wrote: > Thanks for a quick response.. > But the sinks written for cloudera flume (i was going through hello world > plugin) use cloudera classes and all.. For FLume-728 the sinks written > (HDFSSink and IRCSink) is different from cloudera's. > CLoudera flume requires cloudera's hadoop to be installed, if i am not > wrong? > > My idea is to collect the events from Flume agent and send to Esper, where > i will query the streams using EPL. > Can you please tell me what all things are required to write a custom > sink. i mean what needs to be done so that Flume node recognizes the sink. > > Thanks > > Regards, > Som > > > On Wed, Mar 28, 2012 at 1:06 PM, alo alt <[email protected]> wrote: > >> Hi, >> >> Cloudera uses in 3u3 Flume-798 >> >> Apache flume isn't different, you can build it among your installation >> and tweak the configs to match them or use the binary distribution: >> https://github.com/cloudera/flume/downloads >> >> As you read, apache flume 1.1.0 was just released: >> http://www.apache.org/dyn/closer.cgi/incubator/flume/ >> >> You have to write plugins for sinks, here you found some created from the >> community: >> https://cwiki.apache.org/FLUME/flume-plugins.html >> >> And the docs: >> >> http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins >> >> Q3 - I don't know, if you use serialized data whats with avro? >> >> - Alex >> >> -- >> Alexander Lorenz >> http://mapredit.blogspot.com >> >> On Mar 28, 2012, at 5:49 AM, shekhar sharma wrote: >> >> > Hello every one, >> > I am trying to integrate FLume and Esper by writing a custom sink for >> Esper. But as per the documentation , i am unable to do so. i have tried >> the hello world plugin, but that one also not working. >> > I am using FLume-728. >> > I have few question: >> > >> > Q1. Is Apache Flume is different from Cloudera's Flume? >> > If i am using Cloudera's FLume, then does it have any dependency >> that i have to use CLoudera's distribution of Hadoop. >> > >> > Q2. What are the necessary steps for writing custom sink ? >> > >> > >> > Q3. I have implemented a POC in Esper, and it takes POJO (Plain old >> java objects), XML or Map as events. Now if i try integrating both, how >> feasible it is? >> > >> > Regards, >> > Som Shekhar Sharma >> > >> > >> >> >
