----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4662/#review6726 -----------------------------------------------------------
Ship it! +1 Please attach the patch to the jira. - Arvind On 2012-04-06 01:07:38, Prasad Mujumdar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/4662/ > ----------------------------------------------------------- > > (Updated 2012-04-06 01:07:38) > > > Review request for Flume. > > > Summary > ------- > > The sink process() keep tracks of the buckets opened during the transaction. > At the end of transaction, we need to flush all the buckets that has pending > data. This is required in order to ensure that the data removed from channel > should be safely in HDFS during commit. > Currently the files are tracked only when they are created and also getting > closed during the cleanup instead of flush. > > The fix is to track buckets every time they are written to in the current > transaction. Also buckets with pending data should be flushed instead of > close. > > > This addresses bug FLUME-1104. > https://issues.apache.org/jira/browse/FLUME-1104 > > > Diffs > ----- > > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 114682a > > flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java > 6ff3737 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java > 7a94f97 > > Diff: https://reviews.apache.org/r/4662/diff > > > Testing > ------- > > > Thanks, > > Prasad > >
