Hi,
Thanks for your help and suggestions.
Unfortunately, the problem happens during the parsing stage, before any
steps are run. Something is going wrong when Pig tries to turn the query
into a LogicalPlan.
To see for yourself, try running the query below in Grunt. Doesn't matter if
you are running in "-x local" or "-x mapreduce" mode:
A = LOAD 'ignored' AS (ignored, kids:bag{kid:tuple(name, age)});
B = FOREACH A {
filteredkids = FILTER kids BY age != 1;
sortedkids = ORDER filteredkids BY age;
GENERATE sortedkids;
}
DUMP B;
You should see:
2010-02-09 11:54:40,065 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1026: Attempt to fetch field 1 from schema of size 1
When I split up the foreach block it seems to work correctly, leading me to
believe that this is a bug, just want someone to verify as I'm relatively
new to Pig:
A = LOAD 'ignored' AS (ignored, kids:bag{kid:tuple(name, age)});
B = FOREACH A {
filteredkids = FILTER kids BY age != 1;
GENERATE $0, filteredkids;
}
C = FOREACH B {
sortedkids = ORDER $1 BY age;
GENERATE $0, sortedkids;
}
DUMP C;
Best,
Rusty
On Sun, Feb 7, 2010 at 11:00 PM, Rekha Joshi <[email protected]> wrote:
> What might possibly be happening is you get empty bags after filtering.Try
> filtering them out before trying the order.Try checking the describe/dump of
> filtered data.
> And at what stage are you meaning which 'age' element is not very clear
> here.
>
> Cheers,
> /R
>
> On 2/7/10 10:32 PM, "Rusty Klophaus" <[email protected]> wrote:
>
> Hello,
>
> I am trying to FILTER and then ORDER an inner bag, for example:
>
> A = LOAD ...blah... AS (first, last, age, kids:bag{kid:tuple(name, age)});
> B = FOREACH A {
> filteredkids = FILTER kids BY age != 1;
> sortedkids = ORDER filteredkids BY age;
> GENERATE sortedkids;
> }
> DUMP B;
>
> When I run this, I get a stacktrace with a root error of:
>
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1026:
> Attempt to fetch field 1 from schema of size 1
> at org.apache.pig.impl.logicalLayer.schema.Schema.getField(Schema.java:749)
> at
>
> org.apache.pig.impl.logicalLayer.LOProject.getFieldSchema(LOProject.java:289)
>
> $ pig --version
> Apache Pig version 0.5.0 (r829623)
> compiled Oct 25 2009, 18:58:38
>
> If I run just the FILTER or ORDER operation in the inner bag, it works. But
> when I put them both in there, it fails.
>
> Has anybody else experienced this problem? Am I doing something wrong?
>
> Thanks,
> Rusty Klophaus
>
>
