Did you take a look at TOMAP function? Prashant
Sent from my iPhone On Dec 17, 2011, at 2:13 PM, Cameron Gandevia <[email protected]> wrote: > Thanks the cast actually works but now i end up with a tuple that i cant > figute out how to put back into a map > On Dec 17, 2011 1:59 PM, "Jonathan Coveney" <[email protected]> wrote: >> >> I believe you have to actually cast it. This is sort of a weird, known >> issue in pig: doing name:type doesn't actually do the cast. >> >> 2011/12/16 Cameron Gandevia <[email protected]> >> >>> Hi >>> >>> I would like to load Json data remove any duplicates and write it back > as >>> json. I am using the elephant-bird json libraries but I cant figure out > how >>> to project a map. >>> >>> DEFINE JsonLoader com.twitter.elephantbird.pig8.load.LzoJsonLoader(); >>> DEFINE JsonStorage com.twitter.elephantbird.pig8.store.LzoJsonStorage(); >>> raw = LOAD '$INPUT' USING JsonLoader AS (json:map[]); >>> logs = FOREACH raw GENERATE json#'host as host:chararray, json#'body' as >>> body:chararray; >>> dedupped = DISTINCT logs; >>> STORE dedupped INT '$OUTPUT' USING JsonStorage(); >>>
