> On Sept. 17, 2012, 5:58 p.m., Brock Noland wrote: > > flume-ng-core/src/main/java/org/apache/flume/sink/AbstractBucketWriter.java, > > line 84 > > <https://reviews.apache.org/r/7131/diff/1/?file=155840#file155840line84> > > > > This is a hadoop issue and file systems which other sinks such as > > Rolling File sink won't have such an issue. Should this code in a class > > related to hadoop?
agreed > On Sept. 17, 2012, 5:58 p.m., Brock Noland wrote: > > flume-ng-core/src/main/java/org/apache/flume/sink/FileBucketWriter.java, > > line 41 > > <https://reviews.apache.org/r/7131/diff/1/?file=155842#file155842line41> > > > > I think we should be using a File object to get the directory here. agreed > On Sept. 17, 2012, 5:58 p.m., Brock Noland wrote: > > flume-ng-core/src/main/java/org/apache/flume/sink/FileBucketWriter.java, > > line 42 > > <https://reviews.apache.org/r/7131/diff/1/?file=155842#file155842line42> > > > > We should do an assertion or log something if mkdirs fails. agreed - Ted ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7131/#review11619 ----------------------------------------------------------- On Sept. 17, 2012, 3:49 p.m., Ted Malaska wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7131/ > ----------------------------------------------------------- > > (Updated Sept. 17, 2012, 3:49 p.m.) > > > Review request for Flume. > > > Description > ------- > > OK, first note this is still a work in progress, but I'm at the point where I > need feedback. > > Initial goal was to allow RollingFileSink to do Escape Sequence the same way > as HDFSEventSink. > > In order to reuse code, my change attempt to reuse bucketWriter and > BucketWriterLinkedHashMap. > > BucketWriter has been broken up into three files. > > > AbstractBucketWriter : Contains all the rollover and escape sequence logic. > > BucketWriter: Contains the IO logic to HDFS > > FileBucketWriter: contains the IO logic to normal files > > I also added a couple unit tests to RollingFileSink for new types of roll > overs and escape sequences. > > > This addresses bug FLUME-1295. > https://issues.apache.org/jira/browse/FLUME-1295 > > > Diffs > ----- > > flume-ng-core/src/main/java/org/apache/flume/sink/AbstractBucketWriter.java > PRE-CREATION > > flume-ng-core/src/main/java/org/apache/flume/sink/BucketWriterLinkedHashMap.java > PRE-CREATION > flume-ng-core/src/main/java/org/apache/flume/sink/FileBucketWriter.java > PRE-CREATION > flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java > be640bb > flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java > 07fa644 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java > 6408eb9 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 9a76ecb > > Diff: https://reviews.apache.org/r/7131/diff/ > > > Testing > ------- > > > Thanks, > > Ted Malaska > >
