You would want to replicate events to two channels and have something
like a file sink pull from the second channel. You may want to look at
http://flume.apache.org/FlumeUserGuide.html#flume-channel-selectors
As to using headers to determine path, you use the hdfs sink but point
it to the local filesystem. The HDFS sink allows header substitution in
the path.
On 08/11/2013 04:49 PM, Anat Rozenzon wrote:
Hi,
I'm looking for a way to create a sink that will simply 'backup' the
files I'm sending through an Avro Source.
I have in the headers the file name, host name etc.
Is there a way to do store each line in a file according to these headers?
For example store events in a file called
orig_hostname.orig_filename.timestamp, e.g.:
myhost.messages.log.1.135678263178
Thanks
Anat
