Thanks the cast actually works but now i end up with a tuple that i cant
figute out how to put back into a map
On Dec 17, 2011 1:59 PM, "Jonathan Coveney" <[email protected]> wrote:
>
> I believe you have to actually cast it. This is sort of a weird, known
> issue in pig: doing name:type doesn't actually do the cast.
>
> 2011/12/16 Cameron Gandevia <[email protected]>
>
> > Hi
> >
> > I would like to load Json data remove any duplicates and write it back
as
> > json. I am using the elephant-bird json libraries but I cant figure out
how
> > to project a map.
> >
> > DEFINE JsonLoader com.twitter.elephantbird.pig8.load.LzoJsonLoader();
> > DEFINE JsonStorage com.twitter.elephantbird.pig8.store.LzoJsonStorage();
> > raw = LOAD '$INPUT' USING JsonLoader AS (json:map[]);
> > logs = FOREACH raw GENERATE json#'host as host:chararray, json#'body' as
> > body:chararray;
> > dedupped = DISTINCT logs;
> > STORE dedupped INT '$OUTPUT' USING JsonStorage();
> >
