When you load the data here you define the schema of everything but what's
in props:
data = LOAD 'test2.txt' AS (etype:chararray, eid:long, props:[{}]);I think what's in props then is defaulted to a bytearray, and I would guess it's complaining about this line: movie_actors_flattenlist = FOREACH actors_map GENERATE etype, eid, FLATTEN(actors) as (actor_etype:chararray,actor_eid:long); If I had to guess I would say try defining the nested schema of props and see if that works. On Sun, Mar 16, 2014 at 2:43 PM, Davide Brambilla < [email protected]> wrote: > Hi, > I've the following problem with a pig script but I can't figure out > where the problem is. > Please can you help me ? > > Thanks > > Davide B. > > *PIG script* > data = LOAD 'test2.txt' AS (etype:chararray, eid:long, props:[{}]); > actors_map_all = FOREACH data GENERATE etype, eid, props#'actors' as > actors; > actors_map = FILTER actors_map_all BY actors IS NOT NULL; > /*DUMP actors_map;*/ > movie_actors_flattenlist = FOREACH actors_map GENERATE etype, eid, FLATTEN > (actors) as (actor_etype:chararray,actor_eid:long); > movie_actors_flattenlist_gen = FOREACH movie_actors_flattenlist GENERATE > etype, eid, actor_etype, (long)actor_eid; > /*DUMP actors_list_all;*/ > /*DUMP movie_actors_flattenlist;*/ > persons = FILTER data BY etype == 'person'; > movie_actors_flattenlist_expanded = JOIN movie_actors_flattenlist_gen BY > (long)actor_eid, persons BY (long)eid; > DUMP movie_actors_flattenlist_expanded; > > *test2.txt* > movie 1001 [actors#{(person,4001),(person,4002)}] > person 4001 [name#{(Bob)}] > person 4002 [name#{(Alice)}] > > *error* > java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be > cast to java.lang.Long > at > > org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:118) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:111) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) > > > ---------------------------------------------------------------------------------------- > Davide Brambilla > ContentWise R&D > Moviri > [email protected] > phone: +39 02 49517001 mobile: 345 71 13 800 > Moviri S.p.A. - Via Schiaffino, 11 - 20158 Milano (MI) - ITALY >
