Have you looked at some of the test classes? That may be a good way to see how you can accomplish this with straight java.
https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java On Tue, May 20, 2014 at 6:46 AM, Jay Vyas <[email protected]> wrote: > Hi flume ! > > I'd like to implement a simple flume sink: > > agent.channels.memory-channel.type = memory > agent.channels.memory-channel.capacity = 2000 > agent.channels.memory-channel.transactionCapacity = 100 > agent.sources.tail-source.type = exec > agent.sources.tail-source.command = tail -F /tmp/flume-smoke > agent.sources.tail-source.channels = memory-channel > > agent.sinks.log-sink.channel = memory-channel > agent.sinks.log-sink.type = logger > > # Define a sink that outputs to the DFS > agent.sinks.hdfs-sink.channel = memory-channel > agent.sinks.hdfs-sink.type = hdfs > agent.sinks.hdfs-sink.hdfs.path = myfs:///tmp/flume-test > agent.sinks.hdfs-sink.hdfs.fileType = DataStream > > > # activate the channels/sinks/sources > agent.channels = memory-channel > agent.sources = tail-source > agent.sinks = log-sink hdfs-sink > > > But id like to do this dynamically, i.e. without having to package any > static text files. > > Is there a simple way to > > 1) Create the above sink definition and > 2) To start something like "timeout 60 flume-ng agent --conf ./flume/conf/ > -f ./flume/conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent" > > All in Java, ? > > -- > Jay Vyas > http://jayunit100.blogspot.com >
