Lets say you have a directory /mydata of files containing rows in the
format below:
{ "_id" : "foo.com", "categories" : [], "h1" : { "bar==" : { "first" :
1281916800, "last" : 1316995200 }, "foo==" : { "first" : 1281916800, "last"
: 1316995200 } }, "name2" : [ "foobarl.com", "foobar2.com" ], "rep" : null }
Do:
l = load '/mydata' using org.nts.pigutils.udfs.JSONLoader();
-- to get the "_id" value and the "bar=="."first", and the name2 value
value do:
r = foreach l generate m#'_id' as id, m#'bar=='#'first' as bar_first,
m#'name2' as names;
On Fri, Aug 30, 2013 at 9:32 AM, jamal sasha <[email protected]> wrote:
> Hi Gerrit. Can you please help me with an example?
>
>
>
> On Fri, Aug 30, 2013 at 8:48 AM, Gerrit Jansen van Vuuren <
> [email protected]> wrote:
>
> > These are converted to tuples, I could've made them bags but thought
> tuples
> > repesent arrays more generically.
> > On 30 Aug 2013 17:36, "Zhu Wayne" <[email protected]> wrote:
> >
> > > how do you deal with JSON array or list of elements?
> > >
> > >
> > > On Fri, Aug 30, 2013 at 9:45 AM, Gerrit Jansen van Vuuren <
> > > [email protected]> wrote:
> > >
> > > > try using my UDF:
> > > > https://github.com/gerritjvv/pigutils/tree/master/pigudfs/udfs
> > > >
> > > > it loads json as a map, and all nested objects as maps. I've not
> > released
> > > > the jar yet so you'll need to compile a jar from source.
> > > >
> > > > here's an example of how to use it:
> > > >
> > > > l = load 'myfile.json.gz' using org.nts.pigutils.udfs.JSONLoader();
> > > > r = foreach l generate m#'user'#'age', m#'name';
> > > >
> > > >
> > > >
> > > > On Thu, Aug 29, 2013 at 3:19 PM, jamal sasha <[email protected]>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I have json file in follwoing format:
> > > > > { "_id" : "foo.com", "categories" : [], "h1" : { "bar==" : {
> > "first" :
> > > > > 1281916800, "last" : 1316995200 }, "foo==" : { "first" :
> 1281916800,
> > > > "last"
> > > > > : 1316995200 } }, "name2" : [ "foobarl.com", "foobar2.com" ],
> "rep"
> > :
> > > > > null }
> > > > > So, how do i parse this json in pig..
> > > > >
> > > > > also, the categories and rep can have some char in it..and might
> not
> > be
> > > > > always empty.
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > >
> > > > > This message may contain confidential and/or privileged
> information.
> > If
> > > > it
> > > > > has
> > > > > been sent to you in error, please reply to advise the sender of the
> > > error
> > > > > and
> > > > > then immediately delete this message.
> > > >
> > >
> > >
> > >
> > > --
> > > Wayne Zhu
> > > 847-282-0596 (Google Voice)
> > >
> > >
> > >
> > > This message may contain confidential and/or privileged information. If
> > it
> > > has
> > > been sent to you in error, please reply to advise the sender of the
> error
> > > and
> > > then immediately delete this message.
> >
>
>
>
> This message may contain confidential and/or privileged information. If it
> has
> been sent to you in error, please reply to advise the sender of the error
> and
> then immediately delete this message.
>
