Hi Mike, you have to have events each containing a "timestamp" header for that functionality to work.
Mike On Tuesday, May 15, 2012 at 8:37 AM, Mike M wrote: > I'm running into an issue where trying to use an escape sequence with > an HDFS sink causes an exception. For example, if I have the sink > configured like: > > tail-nginx.sinks.HDFS.hdfs.path = hdfs://localhost:8020/flume/%Y/%m/%d/ > > I see the following exception showing up thousands of times: > > 2012-05-14 16:20:15,730 INFO source.ExecSource: Exec source starting > with command:tail -F /var/log/nginx/localhost.access.log > 2012-05-14 16:20:15,743 ERROR hdfs.HDFSEventSink: process failed > java.lang.NumberFormatException: null > at java.lang.Long.parseLong(Long.java:404) > at java.lang.Long.valueOf(Long.java:540) > at > org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:185) > at > org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:219) > at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:344) > at > org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) > at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) > at java.lang.Thread.run(Thread.java:722) > 2012-05-14 16:20:15,745 ERROR flume.SinkRunner: Unable to deliver > event. Exception follows. > org.apache.flume.EventDeliveryException: java.lang.NumberFormatException: null > at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:407) > at > org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) > at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) > at java.lang.Thread.run(Thread.java:722) > Caused by: java.lang.NumberFormatException: null > at java.lang.Long.parseLong(Long.java:404) > at java.lang.Long.valueOf(Long.java:540) > at > org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:185) > at > org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:219) > at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:344) > ... 3 more > > I've tried various methods of quoting the escape sequences, to no > avail. When the escape sequences are removed, Flume works as > expected. Is this just a configuration issue on my side, or is there > possibly a bug? I'm running Flume-NG 1.1.0 packaged with CDH3u4. > > Thanks! > > Mike
