What Wolfgang said :) I'd be interested in hearing how the File Source is different from or better than Exec Source with tail -F or https://issues.apache.org/jira/browse/FLUME-2344 - do you know?
Thanks, Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Fri, May 2, 2014 at 5:18 PM, Israel Ekpo <[email protected]> wrote: > Flume Community, > > I created a Flume Plugin with multiple components that complements the > current version of Apache Flume. > > This was necessary as part of a personal project as I working on. > > It is code named - Flume Jambalaya > > Jambalaya is a standalone Apache Flume plugin that contains a variety of > sources, interceptors, channels, sinks, serializers and other components > designed to extend the Flume architecture. It has been released under the > Apache License version 2.0 > > https://github.com/aicer/flume-jambalaya > > It currently contains: > > (a) File Source - This source lets you ingest data by tailing files from a > specific path > (b) ElasticSearch HTTP Sink - This sink sends events to an ElasticSearch > cluster via HTTP with no dependency on the ElasticSearch versions between > Flume and the Server cluster. > (c) DateInterceptor - The date interceptor is used for parsing dates from > fields and using that date or timestamp as the timestamp for the Flume > event. > (d) Grok Interceptor - allows you to extract structured data from > unstructured text and inject them as headers into the event > > Sample configuration files are available here > > > https://github.com/aicer/flume-jambalaya/tree/master/sample-configuration-files > > I did not realize that the Flume trunk already has a HTTP Sink for > ElasticSearch so you can decide whether or not to use the sink that comes > with it > > I am still testing and integrating the various components. > > Please check it out when you get a chance and send me some feedback > > Thanks. >
