----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68179/ -----------------------------------------------------------
Review request for Flume. Repository: flume-git Description ------- When flume starts up faster than hdfs, then hdfs sink will report NullPoint, nothing will sink to hdfs. Patch have been tested in my dev env. flume_1 | 2018-07-25 03:49:06,147 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN - org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:582)] Caught IOException writing to HDFSWriter (opt). Closing file (hdfs://namenode:9000/flume/log00/201807/FlumeData.1532490542456.avro.tmp) and rethrowing exception. flume_1 | 2018-07-25 03:49:06,161 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:393)] Closing hdfs://namenode:9000/flume/log00/201807/FlumeData.1532490542456.avro.tmp flume_1 | 2018-07-25 03:49:06,164 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:447)] process failed flume_1 | java.lang.NullPointerException flume_1 | at org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187) flume_1 | at org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701) flume_1 | at org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:50) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:698) flume_1 | at java.util.concurrent.FutureTask.run(FutureTask.java:266) flume_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) flume_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) flume_1 | at java.lang.Thread.run(Thread.java:748) flume_1 | 2018-07-25 03:49:06,164 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows. flume_1 | org.apache.flume.EventDeliveryException: java.lang.NullPointerException flume_1 | at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:451) flume_1 | at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67) flume_1 | at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) flume_1 | at java.lang.Thread.run(Thread.java:748) flume_1 | Caused by: java.lang.NullPointerException flume_1 | at org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187) flume_1 | at org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323) flume_1 | at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701) Diffs ----- flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AvroEventSerializer.java 32317424 Diff: https://reviews.apache.org/r/68179/diff/1/ Testing ------- Thanks, wang zheng
