Of course not. That would have been the smart thing to do. Both suggestions fixed the problem and it all works as expected. Thank you, Brock, for the quick responses.
-Chris -----Original Message----- From: Brock Noland [mailto:[email protected]] Sent: Friday, December 14, 2012 4:03 PM To: [email protected] Subject: Re: Flume/HDFS Encoding Did you fix the parameter naming problem I described in the earlier message? On Fri, Dec 14, 2012 at 2:59 PM, Cormier, Christopher <[email protected]> wrote: > Thanks Brock, > When I run it as a DataStream I still get some weird characters between > records. > > [DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬ ;) > [DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬ > ;) > Î[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬ > > > ;0 > > > > ½[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬ > ;0 > :[DATA_HERE] > > I was hoping to avoid the ÿÿÿÿ and spaces (I'm assuming they're characters > that are encoded such that -cat won't show them). > > Any thoughts? > > Thanks again, > > Chris > > -----Original Message----- > From: Brock Noland [mailto:[email protected]] > Sent: Friday, December 14, 2012 3:52 PM > To: [email protected] > Subject: Re: Flume/HDFS Encoding > > Hi, > > On Fri, Dec 14, 2012 at 2:48 PM, Cormier, Christopher > <[email protected]> wrote: >> SEQ!org.apache.hadoop.io.LongWritableorg.apache.hadoop.io.TextY] >> õpµ^R÷ï³¬Õ >> > > This is a SequenceFile. > >> >> requestToHDFS.sinks.HDFS.hdfs.file.Type = DataStream >> >> # also tried... >> >> #requestToHDFS.sinks.HDFS.hdfs.file.Type = SequenceFile >> > > The parameter is hdfs.fileType. See here: > > http://flume.apache.org/FlumeUserGuide.html#hdfs-sink > > It sounds like you want a text file so you should use DataStream. > > Brock -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
