We use lzo, as do some of our customers. We have not tried Snappy yet, but all feedback I ever saw about it was positive.
Otis ---- Sematext :: http://sematext.com/ :: Solr - Lucene - Hadoop - HBase Hadoop ecosystem search :: http://search-hadoop.com/ >________________________________ >From: Wayne <[email protected]> >To: [email protected] >Sent: Wednesday, September 14, 2011 8:33 AM >Subject: Compression > >I wanted to do a poll on what compression libraries people are using and >why. We currently use lzo but are considering other alternatives for various >reasons. We would like to move to CDH3 but adding lzo ourselves is a hassle >we are not looking to take on. It kind of defeats the purpose os using CDH3 >to begin with. We current run 20.0 append. > >I know there are a lot of variables that affect the best decision, but we >are looking for general trends in the community. > >Is lzo still the most recommended? Is there benefit in using the lzo >professional library and does anyone use this? >Is snappy just as good as lzo and a lot easier to deal with in term of node >build/releases? >Does zlib/gzip have any traction? > >Compression ratios are important but as always performance/speed is our >biggest requirement. What are people using and why? Where is the momentum >going? Compression is a huge benefit of hadoop/hbase and having high >compression ratios with solid performance is a major benefit. > >Any recommendations would be appreciated. > >Thanks. > > >
