[
https://issues.apache.org/jira/browse/FLUME-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010060#comment-14010060
]
Roshan Naik commented on FLUME-2147:
------------------------------------
Rajkumar
it appears that some of the upstream flume agents may not be applying the
timestamp header on the events. you can either ensure they are configred
correctly or enable 'useLocalTimeStamp' setting in the hdfs sink(s).
> Missing headers cause events to become stuck in channel
> -------------------------------------------------------
>
> Key: FLUME-2147
> URL: https://issues.apache.org/jira/browse/FLUME-2147
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Reporter: Jonathan Cooper-Ellis
>
> If a sink expects a header but does not find it, events will become stuck in
> the channel and Flume will log NullPointer and EventDelivery exceptions. In a
> memory channel, this can be fixed by restarting. In a file channel,
> restarting does not cause events to be removed.
> 05 Aug 2013 12:21:09,424 ERROR
> [SinkRunner-PollingRunner-DefaultSinkProcessor]
> (org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver
> event. Exception follows.
> org.apache.flume.EventDeliveryException: java.lang.NullPointerException:
> Expected timestamp in the Flume event headers, but it was null
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:426)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.NullPointerException: Expected timestamp in the Flume
> event headers, but it was null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
> at
> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:356)
> ... 3 more
> 05 Aug 2013 12:21:09,424 ERROR
> [SinkRunner-PollingRunner-DefaultSinkProcessor]
> (org.apache.flume.sink.hdfs.HDFSEventSink.process:422) - process failed
> java.lang.NullPointerException: Expected timestamp in the Flume event
> headers, but it was null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
> at
> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:356)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:662)
> I was using RegexExtractorInterceptor to match timestamp for partitioning in
> with HDFS sink.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
