There isn't a polished utility for this, and there should be. I think it'll be entirely straightforward, depending on your specific requirements.
If you look in org.apache.hadoop.chukwa.util.TempFileUtil.RandSeqFileWriter there's an example of code that writes out a sequence file for test purposes. --Ari On Tue, Jan 19, 2010 at 3:46 PM, Bill Graham <billgra...@gmail.com> wrote: > Hi, > > Is there an easy way (maybe using a utility class or the chukwa API) to > manually create a sequence file of chukwa records from a log file without > the need for HDFS? > > My use case is this: I've got pig unit tests that read input sequence file > input using ChukwaStorage from local disk. I generated these files by > putting data into the cluster an waiting for the data processor to run. > We're looking to change the log format though, and I'd like to be able to > write and run the unit tests without putting the new data into the cluster. > > If there were a command line way that I could do this that would be very > helpful. Or if anyone could point me to the relevant classes, I could write > such a utility and contribute it back. > > thanks, > Bill > -- Ari Rabkin asrab...@gmail.com UC Berkeley Computer Science Department
