My sense is that a) is interesting if it evolves into a capable true native tailer, whereas b) is already available in flume and c) and d) are already available in flume via the MorphlineInterceptor
Wolfgang. On May 3, 2014, at 12:18 AM, Israel Ekpo <[email protected]> wrote: > Flume Community, > > I created a Flume Plugin with multiple components that complements the > current version of Apache Flume. > > This was necessary as part of a personal project as I working on. > > It is code named - Flume Jambalaya > > Jambalaya is a standalone Apache Flume plugin that contains a variety of > sources, interceptors, channels, sinks, serializers and other components > designed to extend the Flume architecture. It has been released under the > Apache License version 2.0 > > https://github.com/aicer/flume-jambalaya > > It currently contains: > > (a) File Source - This source lets you ingest data by tailing files from a > specific path > (b) ElasticSearch HTTP Sink - This sink sends events to an ElasticSearch > cluster via HTTP with no dependency on the ElasticSearch versions between > Flume and the Server cluster. > (c) DateInterceptor - The date interceptor is used for parsing dates from > fields and using that date or timestamp as the timestamp for the Flume event. > (d) Grok Interceptor - allows you to extract structured data from > unstructured text and inject them as headers into the event > > Sample configuration files are available here > > https://github.com/aicer/flume-jambalaya/tree/master/sample-configuration-files > > I did not realize that the Flume trunk already has a HTTP Sink for > ElasticSearch so you can decide whether or not to use the sink that comes > with it > > I am still testing and integrating the various components. > > Please check it out when you get a chance and send me some feedback > > Thanks. >
