I am working on the pig 08 compatibility layer; it mostly works, fwiw. Converting to Thrift 0.5 would be fairly straightforward; unfortunately the signatures of Thrift messages changed so the code is not entirely backwards compatible. I don't think the changes for what we do with Pig are material.
Are you trying to load protobufs or thrift files, or do you just want Lzo support? If you just want plain text lzo loading, the loaders in the pig-08 branch totally work. Let me know if you have any issues. D On Wed, Jan 12, 2011 at 12:23 PM, Tyler Coffin <[email protected]> wrote: > There's a fork of elephant-bird where pig-8 support is being worked on: > https://github.com/dvryaboy/elephant-bird/tree/pig-08 > > I haven't given it a shot yet. > > -----Original Message----- > From: Evert Lammerts [mailto:[email protected]] > Sent: January 12, 2011 15:10 > To: '[email protected]' > Subject: LZO & Pig (Elephantbird?) > > Hello list, > > I've installed the LZO codecs (https://github.com/kevinweil/hadoop-lzo) > and > now I'm looking into using LZO in Pig. Elephant Bird > (https://github.com/kevinweil/elephant-bird) seems to provide some nice > prefab loaders, but it's requirements do not fit out Hadoop installation > (we're on CDH3b2 with Pig 0.7, EB cannot be used with anything > 0.6). Also > the need for Thrift 0.2 is unclear to me - Thrift is now at 0.5. > > Now I did find this project, http://code.google.com/p/hadoop-gpl-packing/, > saying EB can handle even Pig 0.8. This confuses me - can I or can I not > use > Elephant Bird with Pig 0.7, or even upgrade to Pig 0.8? > > Since EB is probably not an option, does anybody have some pointers on how > to use LZO'ed files with Pig? > > Thanks! > > Evert Lammerts > > --------------------------------------------------------------------- > This transmission (including any attachments) may contain confidential > information, privileged material (including material protected by the > solicitor-client or other applicable privileges), or constitute non-public > information. Any use of this information by anyone other than the intended > recipient is prohibited. If you have received this transmission in error, > please immediately reply to the sender and delete this information from your > system. Use, dissemination, distribution, or reproduction of this > transmission by unintended recipients is not authorized and may be unlawful. >
