If possible I want to use the official pig distribution, but the company's hadoop (from hadoop 0.19) backend has changed a lot of api from the official one which I can't get the documentaion.
I tried to fix the imcompatial problem by making pig 0.9.1 work with official hadoop 0.19 ,then work with my company's private hadoop distribution . But I don't where to begin. Any help will be welcome. 2012/1/9 Daniel Dai <[email protected]> > What error did you see when you use official Pig distribution? It's > non-trivial to write a backend even if possible (Especially for 0.5, it > would be hard to get help from the community) > > On Mon, Jan 9, 2012 at 12:16 AM, devdoer bird <[email protected]> wrote: > > > So I think Streaming backend may be a good solution for this situation. > > > > 2012/1/9 devdoer bird <[email protected]> > > > > > The reason I decide use Streaming as backend is that In my company the > > > hadoop has been modified so it might not be compatible with the > official > > > hadoop distribution. And I can't make the pig run on our private > hadoop > > > distribution. > > > > > > > > > 2012/1/9 Daniel Dai <[email protected]> > > > > > >> Pig do have a abstraction layer in execution engine. But that is > mostly > > a > > >> legacy of early versions. In recent development, we never keep > platform > > >> neutral in mind so I don't know how reliable this interface is. Can > you > > >> elaborate your idea so we may find a better solution? > > >> > > >> Daniel > > >> > > >> On Sun, Jan 8, 2012 at 7:50 PM, devdoer bird <[email protected]> > > wrote: > > >> > > >> > HI: > > >> > > > >> > I want to implement a new pig backend . Can I replache the hadoop > > >> backend > > >> > with a hadoop--streaming only backend? > > >> > > > >> > I decide to use streaming to implement backend.storage and backend > > >> > .executionengine interface , but I want to know whether it's a > right > > >> way > > >> > to do so. > > >> > > > >> > Thanks. > > >> > > > >> > > > > > > > > >
