I tested it and map output compression works just fine. Thanks Alan. Quick follow-up question. I am trying to store some files ( which are currently gzip compressed ) as SequenceFiles (BLOCK compressed using Gzip). Can you please point me to any documentation that I could use to read/write such sequence files with Pig? SequenceFileLoader seems to be an option but I could not find any documentation around it.
-Prashant On Fri, Nov 18, 2011 at 8:29 AM, Alan Gates <[email protected]> wrote: > > On Nov 17, 2011, at 9:02 PM, Prashant Kommireddi wrote: > > > I understand Pig supports only Gzip and Bzip compression algorithms. > Would > > it be fine if I set map output compression (between map and reduce) to > > SnappyCodec? I am guessing this should not be a problem. > > AFAIK it would be fine, but we haven't tested it. > > > > > Also, is there a plan to add support for SnappyCodec (Load snappy > > compressed files)? > > We would like to do that, but haven't gotten to it yet. You can watch > https://issues.apache.org/jira/browse/PIG-2319 for progress. > > Alan. > > > > > Thanks, > > Prashant > >
