Alright, thanks! I'm going to give that a try and will let you know how it goes.
Cheers, Evert ----- Reply message ----- From: "Dmitriy Ryaboy" <[email protected]> Date: Thu, Jan 13, 2011 8:27 pm Subject: LZO & Pig (Elephantbird?) To: "[email protected]" <[email protected]> Depends on your definition of ready. I haven't put it into production and there's a bit of clean-up left, but as far as I know there are no critical bugs; just some interface issues which are shared between the current master and the version for Pig 0.8. Pig 0.8 should work with Hadoop 0.20.1; I've been working with it under CDH2 (which is a patched 0.20.1). -Dmitriy On Thu, Jan 13, 2011 at 7:43 AM, Evert Lammerts <[email protected]>wrote: > > Are you trying to load protobufs or thrift files, or do you just want > > Lzo > > support? > > Protobufs would be nice, but Elephant Bird is not ready yet for Pig 0.7 / > 0.8, right? > > > If you just want plain text lzo loading, the loaders in the > > pig-08 > > branch totally work. > > Does Pig 0.8 work with Hadoop 0.20.1 as well? > > Thanks for the support! > Evert > > > > > Let me know if you have any issues. > > > > D > > > > On Wed, Jan 12, 2011 at 12:23 PM, Tyler Coffin <[email protected]> wrote: > > > > > There's a fork of elephant-bird where pig-8 support is being worked > > on: > > > https://github.com/dvryaboy/elephant-bird/tree/pig-08 > > > > > > I haven't given it a shot yet. > > > > > > -----Original Message----- > > > From: Evert Lammerts [mailto:[email protected]] > > > Sent: January 12, 2011 15:10 > > > To: '[email protected]' > > > Subject: LZO & Pig (Elephantbird?) > > > > > > Hello list, > > > > > > I've installed the LZO codecs (https://github.com/kevinweil/hadoop- > > lzo) > > > and > > > now I'm looking into using LZO in Pig. Elephant Bird > > > (https://github.com/kevinweil/elephant-bird) seems to provide some > > nice > > > prefab loaders, but it's requirements do not fit out Hadoop > > installation > > > (we're on CDH3b2 with Pig 0.7, EB cannot be used with anything > > > 0.6). Also > > > the need for Thrift 0.2 is unclear to me - Thrift is now at 0.5. > > > > > > Now I did find this project, http://code.google.com/p/hadoop-gpl- > > packing/, > > > saying EB can handle even Pig 0.8. This confuses me - can I or can I > > not > > > use > > > Elephant Bird with Pig 0.7, or even upgrade to Pig 0.8? > > > > > > Since EB is probably not an option, does anybody have some pointers > > on how > > > to use LZO'ed files with Pig? > > > > > > Thanks! > > > > > > Evert Lammerts > > > > > > --------------------------------------------------------------------- > > > This transmission (including any attachments) may contain > > confidential > > > information, privileged material (including material protected by the > > > solicitor-client or other applicable privileges), or constitute non- > > public > > > information. Any use of this information by anyone other than the > > intended > > > recipient is prohibited. If you have received this transmission in > > error, > > > please immediately reply to the sender and delete this information > > from your > > > system. Use, dissemination, distribution, or reproduction of this > > > transmission by unintended recipients is not authorized and may be > > unlawful. > > > >
