[
https://issues.apache.org/jira/browse/FLUME-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzheng updated FLUME-3264:
-----------------------------
Release Note: Add null point check at flush(), to get around null point
exception in case flume startup faster than hdfs.
Attachment: FLUME-3264-0.patch
Status: Patch Available (was: Open)
> Null point using hdfs sink when flume startup faster than hdfs
> --------------------------------------------------------------
>
> Key: FLUME-3264
> URL: https://issues.apache.org/jira/browse/FLUME-3264
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.8.0, 1.7.0
> Environment: flume 1.8.0, hdfs 1.9.1.
>
> Reporter: wangzheng
> Priority: Trivial
> Attachments: FLUME-3264-0.patch, FLUME-3264-0.patch
>
>
> When flume starts up faster than hdfs, then hdfs sink will report NullPoint,
> nothing will sink to hdfs. Patch have been tested in my dev env.
>
> flume_1 | 2018-07-25 03:49:06,147
> (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN -
> org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:582)] Caught
> IOException writing to HDFSWriter (opt). Closing file
> (hdfs://namenode:9000/flume/log00/201807/FlumeData.1532490542456.avro.tmp)
> and rethrowing exception.
> flume_1 | 2018-07-25 03:49:06,161
> (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO -
> org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:393)] Closing
> hdfs://namenode:9000/flume/log00/201807/FlumeData.1532490542456.avro.tmp
> flume_1 | 2018-07-25 03:49:06,164
> (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:447)]
> process failed
> flume_1 | java.lang.NullPointerException
> flume_1 | at
> org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187)
> flume_1 | at
> org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701)
> flume_1 | at
> org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:50)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:698)
> flume_1 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> flume_1 | at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> flume_1 | at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> flume_1 | at java.lang.Thread.run(Thread.java:748)
> flume_1 | 2018-07-25 03:49:06,164
> (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to
> deliver event. Exception follows.
> flume_1 | org.apache.flume.EventDeliveryException:
> java.lang.NullPointerException
> flume_1 | at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:451)
> flume_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
> flume_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
> flume_1 | at java.lang.Thread.run(Thread.java:748)
> flume_1 | Caused by: java.lang.NullPointerException
> flume_1 | at
> org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187)
> flume_1 | at
> org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323)
> flume_1 | at
> org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
