Vijay it really depends on what you are doing with LZO. Is it being used for creating splits, map output compression, intermediate files? Also what are you comparing this to? Simple text files, gzip/bzip compressed files?
Sent from my iPhone On Dec 10, 2011, at 11:12 PM, vijaya bhaskar peddinti <[email protected]> wrote: > Dear All, > > I am doing a PoC on Lzo compression with Protobuf using elephant bird and > Pig 0.8.0. I am doing this PoC on cluster of 10 nodes. I have also done > indexing for the Lzo file. i have noticed that there is no performance > improvement when compared with uncompressed data. Does Lzo support is there > for Pig? > > The data size if 1.5GB for the PoC. Pig script is a select query kind of > which reads and writes data using Lzo*ProtoBuf Loader and storage methods. > > Please provide any suggestions and pointer in this regards. > > > thanks and regards, > Vijaya Bhaskar Peddinti
