You can get the original file name (if you're using any of the tail
sources) from the tailSrcFile attribute. We use it like this:
collectorSink("file:///logs/flume/apps/%{application}/access/%Y-%m-%d/%H/",
"%{tailSrcFile}-%{host}-", 300000),
The second argument is the file name that will be used. If you're not
using the tail sources, you could add a custom attribute to your flow
definition that contains the file name.
As for parsing the file name and using that to create the folder names,
this thread
(https://groups.google.com/a/cloudera.org/group/flume-user/msg/9e78675c759223ed)
has an example of using the regex extractor to pull parts of the message
out to use in the collector file name.
Cheers,
Dan
On Wed, 27 Jul 2011 12:03 +0800, "Junxian Yan" <junxian....@gmail.com>
wrote:
> Copy this question to new group
>
> ---------- Forwarded message ----------
> From: Junxian Yan <junxian....@gmail.com>
> Date: Tue, Jul 26, 2011 at 2:59 PM
> Subject: Re: Is it possible to get the file name from flume agent
> To: Flume Users <flume-u...@cloudera.org>
>
>
> Or maybe a more specific description for my question:
> Is it possible to get the original file name from flume agent output
>
> R
>
>
> On Tue, Jul 26, 2011 at 2:58 PM, Junxian Yan <junxian....@gmail.com>
> wrote:
>
> > Hi Guys
> >
> > I want to customize the collector output file name and the storing path by
> > the log file name.
> > Such as if the log name is a-b.log
> > I want the storing path is like /a/b/a-b.log
> >
> > Is it possible?
> >
> >
> >
>
