Thanks for a quick response.. But the sinks written for cloudera flume (i was going through hello world plugin) use cloudera classes and all.. For FLume-728 the sinks written (HDFSSink and IRCSink) is different from cloudera's. CLoudera flume requires cloudera's hadoop to be installed, if i am not wrong?
My idea is to collect the events from Flume agent and send to Esper, where i will query the streams using EPL. Can you please tell me what all things are required to write a custom sink. i mean what needs to be done so that Flume node recognizes the sink. Thanks Regards, Som On Wed, Mar 28, 2012 at 1:06 PM, alo alt <[email protected]> wrote: > Hi, > > Cloudera uses in 3u3 Flume-798 > > Apache flume isn't different, you can build it among your installation and > tweak the configs to match them or use the binary distribution: > https://github.com/cloudera/flume/downloads > > As you read, apache flume 1.1.0 was just released: > http://www.apache.org/dyn/closer.cgi/incubator/flume/ > > You have to write plugins for sinks, here you found some created from the > community: > https://cwiki.apache.org/FLUME/flume-plugins.html > > And the docs: > > http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins > > Q3 - I don't know, if you use serialized data whats with avro? > > - Alex > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Mar 28, 2012, at 5:49 AM, shekhar sharma wrote: > > > Hello every one, > > I am trying to integrate FLume and Esper by writing a custom sink for > Esper. But as per the documentation , i am unable to do so. i have tried > the hello world plugin, but that one also not working. > > I am using FLume-728. > > I have few question: > > > > Q1. Is Apache Flume is different from Cloudera's Flume? > > If i am using Cloudera's FLume, then does it have any dependency > that i have to use CLoudera's distribution of Hadoop. > > > > Q2. What are the necessary steps for writing custom sink ? > > > > > > Q3. I have implemented a POC in Esper, and it takes POJO (Plain old > java objects), XML or Map as events. Now if i try integrating both, how > feasible it is? > > > > Regards, > > Som Shekhar Sharma > > > > > >
