Thanks Brock,
When I run it as a DataStream I still get some weird characters between records.
[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬ ;)
[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬
;)
Î[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬
;0
½[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬
;0
:[DATA_HERE]
I was hoping to avoid the ÿÿÿÿ and spaces (I'm assuming they're characters that
are encoded such that -cat won't show them).
Any thoughts?
Thanks again,
Chris
-----Original Message-----
From: Brock Noland [mailto:[email protected]]
Sent: Friday, December 14, 2012 3:52 PM
To: [email protected]
Subject: Re: Flume/HDFS Encoding
Hi,
On Fri, Dec 14, 2012 at 2:48 PM, Cormier, Christopher
<[email protected]> wrote:
> SEQ!org.apache.hadoop.io.LongWritableorg.apache.hadoop.io.TextY]
> õpµ^R÷ﳬÕ
>
This is a SequenceFile.
>
> requestToHDFS.sinks.HDFS.hdfs.file.Type = DataStream
>
> # also tried...
>
> #requestToHDFS.sinks.HDFS.hdfs.file.Type = SequenceFile
>
The parameter is hdfs.fileType. See here:
http://flume.apache.org/FlumeUserGuide.html#hdfs-sink
It sounds like you want a text file so you should use DataStream.
Brock
