Run a standard job before. Look at the summary data. Run the job again after the changes and look at the summary.
You should see less file system bytes written from the map stage. Sorry, might be most obvious in shuffle bytes. I don't have a terminal in front of me right now. James Sent from my mobile. Please excuse the typos. On 2011-02-24, at 8:22 AM, Marc Sturlese <[email protected]> wrote: > > Hey there, > I am using hadoop 0.20.2. I 've successfully installed LZOCompression > following these steps: > https://github.com/kevinweil/hadoop-lzo > > I have some MR jobs written with the new API and I want to compress > intermediate data. > Not sure if my mapred-site.xml should have the properties: > > <property> > <name>mapred.compress.map.output</name> > <value>true</value> > </property> > <property> > <name>mapred.map.output.compression.codec</name> > <value>com.hadoop.compression.lzo.LzoCodec</value> > </property> > > or: > > <property> > <name>mapreduce.map.output.compress</name> > <value>true</value> > </property> > <property> > <name>mapreduce.map.output.compress.codec</name> > <value>com.hadoop.compression.lzo.LzoCodec</value> > </property> > > How can I check that the compression is been applied? > > Thanks in advance > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Check-lzo-is-working-on-intermediate-data-tp2567704p2567704.html > Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
