Hi Som, I'd be interested also in connection flume against Esper. Would you mind sharing your code if it's meant to be open source?
Thanks, Dennis Von: shekhar sharma <[email protected]<mailto:[email protected]>> Antworten an: <[email protected]<mailto:[email protected]>> Datum: Wed, 28 Mar 2012 15:45:35 +0530 An: <[email protected]<mailto:[email protected]>> Betreff: Re: Integrating Flume a Esper Hel, i have written a custom sink for Esper, can you please how would i make FLume to recognize this is my new sink..Do i have to specify in configuration file. Can you please help me? Regards, Som On Wed, Mar 28, 2012 at 1:56 PM, shekhar sharma <[email protected]<mailto:[email protected]>> wrote: Thanks for a quick response.. But the sinks written for cloudera flume (i was going through hello world plugin) use cloudera classes and all.. For FLume-728 the sinks written (HDFSSink and IRCSink) is different from cloudera's. CLoudera flume requires cloudera's hadoop to be installed, if i am not wrong? My idea is to collect the events from Flume agent and send to Esper, where i will query the streams using EPL. Can you please tell me what all things are required to write a custom sink. i mean what needs to be done so that Flume node recognizes the sink. Thanks Regards, Som On Wed, Mar 28, 2012 at 1:06 PM, alo alt <[email protected]<mailto:[email protected]>> wrote: Hi, Cloudera uses in 3u3 Flume-798 Apache flume isn't different, you can build it among your installation and tweak the configs to match them or use the binary distribution: https://github.com/cloudera/flume/downloads As you read, apache flume 1.1.0 was just released: http://www.apache.org/dyn/closer.cgi/incubator/flume/ You have to write plugins for sinks, here you found some created from the community: https://cwiki.apache.org/FLUME/flume-plugins.html And the docs: http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins Q3 - I don't know, if you use serialized data whats with avro? - Alex -- Alexander Lorenz http://mapredit.blogspot.com On Mar 28, 2012, at 5:49 AM, shekhar sharma wrote: > Hello every one, > I am trying to integrate FLume and Esper by writing a custom sink for Esper. > But as per the documentation , i am unable to do so. i have tried the hello > world plugin, but that one also not working. > I am using FLume-728. > I have few question: > > Q1. Is Apache Flume is different from Cloudera's Flume? > If i am using Cloudera's FLume, then does it have any dependency that > i have to use CLoudera's distribution of Hadoop. > > Q2. What are the necessary steps for writing custom sink ? > > > Q3. I have implemented a POC in Esper, and it takes POJO (Plain old java > objects), XML or Map as events. Now if i try integrating both, how feasible > it is? > > Regards, > Som Shekhar Sharma > >
