Hello, We have some more plug-ins available here : https://github.com/figarocms/flume-plugins There are based on Cloudera flume cdh3u3.
Some are very inspire on existing sinks plugins like HbaseSink (backport to depend on maven cdh3u3 with maven), RabbitMQSink (which include fail-over on RMQ cluster) Others are decorators (haproxy, geoip, filter) and a configurable format (json_object) Maybe this could help you for you own plugin. Regards, -- Damien Le 28/03/2012 10:26, shekhar sharma a écrit : > Thanks for a quick response.. > But the sinks written for cloudera flume (i was going through hello > world plugin) use cloudera classes and all.. For FLume-728 the sinks > written (HDFSSink and IRCSink) is different from cloudera's. > CLoudera flume requires cloudera's hadoop to be installed, if i am > not wrong? > > My idea is to collect the events from Flume agent and send to Esper, > where i will query the streams using EPL. > Can you please tell me what all things are required to write a custom > sink. i mean what needs to be done so that Flume node recognizes the sink. > > Thanks > > Regards, > Som > > On Wed, Mar 28, 2012 at 1:06 PM, alo alt <[email protected] > <mailto:[email protected]>> wrote: > > Hi, > > Cloudera uses in 3u3 Flume-798 > > Apache flume isn't different, you can build it among your > installation and tweak the configs to match them or use the binary > distribution: > https://github.com/cloudera/flume/downloads > > As you read, apache flume 1.1.0 was just released: > http://www.apache.org/dyn/closer.cgi/incubator/flume/ > > You have to write plugins for sinks, here you found some created > from the community: > https://cwiki.apache.org/FLUME/flume-plugins.html > > And the docs: > > http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins > > Q3 - I don't know, if you use serialized data whats with avro? > > - Alex > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Mar 28, 2012, at 5:49 AM, shekhar sharma wrote: > > > Hello every one, > > I am trying to integrate FLume and Esper by writing a custom > sink for Esper. But as per the documentation , i am unable to do > so. i have tried the hello world plugin, but that one also not > working. > > I am using FLume-728. > > I have few question: > > > > Q1. Is Apache Flume is different from Cloudera's Flume? > > If i am using Cloudera's FLume, then does it have any > dependency that i have to use CLoudera's distribution of Hadoop. > > > > Q2. What are the necessary steps for writing custom sink ? > > > > > > Q3. I have implemented a POC in Esper, and it takes POJO (Plain > old java objects), XML or Map as events. Now if i try integrating > both, how feasible it is? > > > > Regards, > > Som Shekhar Sharma > > > > > >
signature.asc
Description: OpenPGP digital signature
