thanks! I thought things like timestamps, year etc is evaluated
automatically since that generally is the case in unix for instance "date
+%h%y".
Could you please point me to some examples where I can see how to construct
these path names?
On Tue, Jun 12, 2012 at 8:11 PM, Will McQueen <w...@cloudera.com> wrote:
> Hi Mohit,
>
> I believe that's a documentation error. Saying %{host} is like saying
> "Look at the current event's header list. Look for an header with key named
> 'host'. Then substitute that header's corresponding value here". AFAIK,
> there's no 'host' header by default in flume events.
>
> You could try creating a custom interceptor that adds a 'host' header key
> to each event, along with the corresponding hostname as the header value.
> Note that you can chain interceptors together, like specifying both a
> HostNameInterceptor and TimestampInterceptor.
>
> As an aside, you should be able to include any arbitrary header value in
> your bucket path (eg, %{foo}) by using a combination of inteceptors and
> pucket path escape sequences. Note that time-based escape sequences like %h
> assume that you have a 'timestamp' header in the flume events. Otherwise
> they will probably all evaluate to just empty string.
>
> Cheers,
> Will
>
>
> On Tue, Jun 12, 2012 at 6:04 PM, Mohit Anchlia <mohitanch...@gmail.com>wrote:
>
>> I configured conf to say foo.sinks.hdfsSink.hdfs.path =
>> hdfs://dsdb1:54310/flume/'%{host}'
>>
>> but when I look at the dir name in console and in hdfs it looks like this:
>>
>>
>> 12/06/12 18:01:27 INFO hdfs.HDFSEventSink: Closing
>> hdfs://dsdb1:54310/flume/''/FlumeData
>>
>> 12/06/12 18:01:27 INFO hdfs.BucketWriter: Renaming
>> hdfs://dsdb1:54310/flume/''/FlumeData.1339549272477.tmp to
>> hdfs://dsdb1:54310/flume/''/FlumeData.1339549272477
>>
>>
>>
>> It's missing the hostname.
>>
>>
>>
>
>
