Hey Zheng, To get Kevin Weil's updated LZO work in a nice RPM or DEB package, check out http://github.com/toddlipcon/hadoop-lzo-packager.
Later, Jeff 2009/10/23 Zheng Shao <[email protected]> > Thanks Yongqiang. > > > > My last message was wrong – we had mapred.compress.map.output=false. > > Will try to get LZO on our cluster and see. > > > > Zheng > > *From:* He Yongqiang [mailto:[email protected]] > *Sent:* Friday, October 23, 2009 1:28 PM > *To:* [email protected] > *Subject:* Re: DefaultCodec vs. LZO > > > > I did a search, and find the below two links, hope they are helpful. > http://blog.oskarsson.nu/2009/03/hadoop-feat-lzo-save-disk-space-and.html > http://mail.jabber.org/pipermail/standards/2005-October/008768.html > > I think Hadoop’s DefaultCodec is using native Zlib when possible. > > Thanks, > Yongqiang > On 09-10-23 下午12:43, "Zheng Shao" <[email protected]> wrote: > > We are using Hadoop 0.20 and mapred.map.output.compression.Codec is set to > DefaultCodec. > We tried LZO but the performance seems very similar to DefaultCodec. > > I heard of a lot of good words about LZO. So did anybody compare LZO with > DefaultCodec? > Is there a big difference? > > We are running CentOS release 5.2 (Final). > > Thanks, > Zheng > >
