I don't think that ExecSource was written for that purpose but you *could* do it. Of course the interface might change at any random time since it's not a normal use case.
https://github.com/apache/flume/blob/master/flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java You'd likely just need to create a channel processor https://github.com/apache/flume/blob/master/flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java The embedded agent is much different, but might be worth a look as well: https://github.com/apache/flume/blob/trunk/flume-ng-doc/sphinx/FlumeDeveloperGuide.rst#embedded-agent Brock On Thu, Jun 27, 2013 at 6:02 PM, Otis Gospodnetic < [email protected]> wrote: > Hi Alex, > > Sorry, no, I didn't mean tailing of N files into one file. > > What I'm wondering is whether I could "steal" a few Flume classes that > handle the tailing functionality without having to use all of Flume. > Imagine I want to use Flume's tailing functionality in my app, so I want > to use Flume as a library.... not being familiar with Flume's APIs, imagine > if in my app I wanted to do this: > > FlumeTail ft = new FlumeTail(new MyLineHandler()); > ft.addFile("/tmp/foo.log"); > ft.exec() > > class MyLineHandler extends SomeBaseFlumeLineHandler { > void handleLine(String line) { > // do whatever I want with that 'line' > } > } > > So my questions are: > * Is something like the above even possible? > * Could I borrow a few Flume classes just for doing something like the > above? > > Thanks, > Otis > ---- > Monitoring for Solr / ElasticSearch / HBase / Hadoop - > http://sematext.com/spm > > ------------------------------ > *From:* Alexander Alten-Lorenz <[email protected]> > *To:* [email protected] > *Sent:* Wednesday, June 26, 2013 4:19 AM > *Subject:* Re: Using Flume *only* for tailing? > > Hi Otis, > > Do you mean tailing into one file? Maybe 100 exec sources => Flume => one > text file in a local filesystem? > > - Alex > > On Jun 25, 2013, at 6:11 PM, Otis Gospodnetic <[email protected]> > wrote: > > > Hi, > > > > I'm wondering if Flume (its ExecSource?) can be used separately from > > the rest of Flume? > > > > Specifically, I need to tail a log file from a Java app and then do > > something with each new line. I just want my Java app to get each new > > line from tailing and process it in some custom way. In other words, > > I don't really need Flume to be involved in anything beyond tailing > > files. > > > > Is that doable? > > > > Thanks, > > Otis > > -- > > Solr & ElasticSearch Support -- http://sematext.com/ > > Performance Monitoring -- http://sematext.com/spm > > -- > Alexander Alten-Lorenz > http://mapredit.blogspot.com > German Hadoop LinkedIn Group: http://goo.gl/N8pCF > > > -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
