----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20698/#review41642 -----------------------------------------------------------
flume-ng-doc/sphinx/FlumeUserGuide.rst <https://reviews.apache.org/r/20698/#comment75196> How about: "If set to 1, this sink will not re-try a failed close (due to, for example, NameNode or DataNode failure), and may leave the file in an open state with a .tmp extension" Negative numbers don't really make sense, and I believe the impl treats them the same as 0 so probably documenting that doesn't really add clarity flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java <https://reviews.apache.org/r/20698/#comment75195> What is this testing? Did you mean to assert on retry interval here? pom.xml <https://reviews.apache.org/r/20698/#comment75197> How about a separate patch for this? - Mike Percy On April 28, 2014, 9:36 p.m., Hari Shreedharan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20698/ > ----------------------------------------------------------- > > (Updated April 28, 2014, 9:36 p.m.) > > > Review request for Flume. > > > Bugs: FLUME-2357 > https://issues.apache.org/jira/browse/FLUME-2357 > > > Repository: flume-git > > > Description > ------- > > Much of the size of the patch is due to a couple of file renames. Otherwise > the patch itself is pretty simple. In the Bucketwriter, if a close fails, we > simply reschedule the close to happen sometime later until it finally > succeeds or till we hit a maximum count. Added a test case too. This depends > on the presence of the isFileClosed method in the HDFS client API. If the > method is absent, reattempts are not done. > > > Diffs > ----- > > flume-ng-doc/sphinx/FlumeUserGuide.rst 7b918ed > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AbstractHDFSWriter.java > da0466d > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java > e82d13d > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java > 5518547 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java > e20d1ee > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 4ea78c1 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java > 5fe9f1b > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockDataStream.java > PRE-CREATION > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFileSystem.java > PRE-CREATION > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFileSystemCloseRetryWrapper.java > b5d89e6 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFsDataOutputStream.java > PRE-CREATION > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFsDataOutputStreamCloseRetryWrapper.java > 1d8c140 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java > b7cc586 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java > 87918d1 > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java > 4476530 > pom.xml 2aa0ad1 > > Diff: https://reviews.apache.org/r/20698/diff/ > > > Testing > ------- > > Added new unit test. All current tests pass. > > > Thanks, > > Hari Shreedharan > >
