I'm running into an issue where trying to use an escape sequence with
an HDFS sink causes an exception. For example, if I have the sink
configured like:
tail-nginx.sinks.HDFS.hdfs.path = hdfs://localhost:8020/flume/%Y/%m/%d/
I see the following exception showing up thousands of times:
2012-05-14 16:20:15,730 INFO source.ExecSource: Exec source starting
with command:tail -F /var/log/nginx/localhost.access.log
2012-05-14 16:20:15,743 ERROR hdfs.HDFSEventSink: process failed
java.lang.NumberFormatException: null
at java.lang.Long.parseLong(Long.java:404)
at java.lang.Long.valueOf(Long.java:540)
at
org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:185)
at
org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:219)
at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:722)
2012-05-14 16:20:15,745 ERROR flume.SinkRunner: Unable to deliver
event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.NumberFormatException: null
at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:407)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NumberFormatException: null
at java.lang.Long.parseLong(Long.java:404)
at java.lang.Long.valueOf(Long.java:540)
at
org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:185)
at
org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:219)
at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:344)
... 3 more
I've tried various methods of quoting the escape sequences, to no
avail. When the escape sequences are removed, Flume works as
expected. Is this just a configuration issue on my side, or is there
possibly a bug? I'm running Flume-NG 1.1.0 packaged with CDH3u4.
Thanks!
Mike
