Hi, I am creating o/p files in reducer using my own file name convention. So, using FileSystem APIs I am dumping data in the files. I now want to compress these files while writing so as to write lesser amount of data and also to save the space on HDFS.
So, I tried following options, but none of them worked: - setting the "mapred.output.compress" to true - job.setOutputFormatClass( TextOutputFormat.class); TextOutputFormat.setCompressOutput(job, true); TextOutputFormat.setOutputCompressorClass(job, GzipCodec.class); - I also tried looking into the exiting FileSystem and FileUtil APIs but none of them has an API to write the file in compressed format Can you please suggest how can I achieve the required goal. -- Regards, Piyush Kansal
