You can specify the compression/codec in the file writer (is this what you are asking?).
SequenceFile.createWriter(fs, conf, path, key.getClass(), value.getClass(), SequenceFile.CompressionType.BLOCK, codec); You can also create your own FileOutputFormat. On Mon, Jan 18, 2010 at 12:53 PM, Ted Yu <[email protected]> wrote: > Hi, > mapred.output.compress is set to true in hadoop-site.xml > My question is how can I specify different compression codecs > programmatically ? > > For example, normally the output is gzip compressed. But a small portion of > output needs to be LZO compressed. > > Thanks >
