Ankur commented on PIG-919:

I have seem this issue in other places when the value coming out of a map[] is 
used in a group/cogroup/join. Pig throws a the same error. And Viraj is right, 
explicit casting to chararray alleviates the issue. But this is confusing for 
users. Pig should be converting "NullableText" to "NullableBytesWritable" 
automatically. Here is another sample script that throws an error. Exlicit 
casting to chararray resolves the issue

data = LOAD 'mydata' USING CustomLoader()  AS (f1:double, f2: map[])

dataProjected =  FOREACH data GENERATE f2#'Url' as url, f1 as rank

data2 = LOAD 'urlList' AS (url:bytearray);

grouped = COGROUP data BY url, data2 url Parallel 10;

STORE grouped INTO 'results'

> Type mismatch in key from map: expected 
> org.apache.pig.impl.io.NullableBytesWritable, recieved 
> org.apache.pig.impl.io.NullableText when doing simple group
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>                 Key: PIG-919
>                 URL: https://issues.apache.org/jira/browse/PIG-919
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Viraj Bhat
>             Fix For: 0.3.0
>         Attachments: GenHashList.java, mapscript.pig, mymapudf.jar
> I have a Pig script, which takes in a student file and generates a bag of 
> maps.  I later want to group on the value of the key "name0" which 
> corresponds to the first name of the student.
> {code}
> register mymapudf.jar;
> data = LOAD '/user/viraj/studenttab10k' AS 
> (somename:chararray,age:long,marks:float);
> genmap = foreach data generate flatten(mymapudf.GenHashList(somename,' ')) as 
> bp:map[], age, marks;
> getfirstnames = foreach genmap generate bp#'name0' as firstname, age, marks;
> filternonnullfirstnames = filter getfirstnames by firstname is not null;
> groupgenmap = group filternonnullfirstnames by firstname;
> dump groupgenmap;
> {code}
> When I execute this code, I get an error in the Map Phase:
> ===========================================================================================================
> java.io.IOException: Type mismatch in key from map: expected 
> org.apache.pig.impl.io.NullableBytesWritable, recieved 
> org.apache.pig.impl.io.NullableText
>       at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
>       at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
>       at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:253)
>       at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
>       at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
>       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>       at 
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
> ===========================================================================================================

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to