-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3988/
-----------------------------------------------------------
(Updated 2012-03-23 20:55:21.762184)
Review request for Flume.
Changes
-------
Rebased patch attached. Attaching to JIRA for commit.
Summary
-------
1) All HDFS actions are now done in async mode
2) If an HDFS append timesout, the file is closed and reopened.
3) Batching is now handled by BucketWriter which was always aware of the batch
size.
This addresses bug FLUME-985.
https://issues.apache.org/jira/browse/FLUME-985
Diffs (updated)
-----
flume-ng-sinks/flume-hdfs-sink/pom.xml bef2ca7
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
45769f6
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
1fdaddd
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java
19b2559
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadSeqWriter.java
8a6740f
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadWriterFactory.java
b067c00
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
8fa72a1
Diff: https://reviews.apache.org/r/3988/diff
Testing
-------
1) Unit tests were added for close/reopen scenario.
2) All unit tests pass
3) I manually verified this patch improved FlumeNG's behavior when the datanode
it's writing to is restarted. In the past FlumeNG had to be restarted, now
Flume moves on and starts writing to a new file.
Thanks,
Brock
