Jonathan Cooper-Ellis created FLUME-2147:
--------------------------------------------
Summary: RegexExtractorInterceptor miss causes irrecoverable
IllegalStateException
Key: FLUME-2147
URL: https://issues.apache.org/jira/browse/FLUME-2147
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Reporter: Jonathan Cooper-Ellis
If a sink expects a header but does not find it, events will become stuck in
the channel and Flume will log NullPointer and EventDelivery exceptions. In a
memory channel, this can be fixed by restarting. In a file channel, restarting
does not cause events to be removed.
05 Aug 2013 12:21:09,424 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver event.
Exception follows.
org.apache.flume.EventDeliveryException: java.lang.NullPointerException:
Expected timestamp in the Flume event headers, but it was null
at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:426)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException: Expected timestamp in the Flume
event headers, but it was null
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at
org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
at
org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:356)
... 3 more
05 Aug 2013 12:21:09,424 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.hdfs.HDFSEventSink.process:422) - process failed
java.lang.NullPointerException: Expected timestamp in the Flume event headers,
but it was null
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at
org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
at
org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:356)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
I was using RegexExtractorInterceptor to match timestamp for partitioning in
with HDFS sink.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
