Re: Pig syntax to access fields of records in an array

[ey-chih chow]

I have a Pig script.  The script begins with a load statement that loads
data in an avro file.  The schema of data in the file has a field com that
is defined in the following way in the schema:

{
    "name" : "com",
    "type" : {
      "type" : "array",
      "items" : {
        "type" : "record",
        "name" : "campaignRecord",
        "doc" : "RTB json logs flattened.",
        "fields" : [ {
          "name" : "BIDCOUNT",
          "type" : "int"
        }}
}
}


After the load statement, there is a group-by statement that does a group
by on some other fields.   After the group-by, we have the following
statement:

FOREACH gstmt group AS key,SUM(RTBALLLOGS.com.BIDCOUNT) AS BIDCOUNT;

This statement is not working with the following message when I debug the
script with Eclipse:

Cannot find field BIDCOUNT in com:bag{ARRAY_ELEM:tuple(BIDCOUNT;int))

Thanks.

Ey-Chih


On Thu, Nov 28, 2013 at 9:39 AM, Ruslan Al-Fakikh <metarus...@gmail.com>wrote:

> I think your expression ends up with a bag with just that column. Can you
> give the full context where it is not working?
> 28 нояб. 2013 г. 2:14 пользователь "ey-chih chow" <eyc...@gmail.com>
> написал:
>
> > Hi,
> >
> > We have an Avro file of which a field that is an array of tuples as
> > follows:
> >
> >
> > cam:bag{ARRAY_ELEM:tuple(BIDCOUNT: int, ...
> >
> >
> > I tried to access BIDCOUNT with 'cam.BIDCOUNT'.  It is not working.  Any
> > body knows how to access BIDCOUNT?  Thanks.
> >
> >
> > Ey-Chih Chow
> >
>