[ 
https://issues.apache.org/jira/browse/PIG-746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695067#action_12695067
 ] 

Pradeep Kamath commented on PIG-746:
------------------------------------

The issue is that the combiner code does not handle projection of bags (because 
currently in the pig implementation, projection of bags is not combineable). 
The combiner optimizer code should identify projection of bags in the foreach 
following the group and in such cases decide not to use the combiner. So the 
quick fix is to disable the combiner for this (till projection of bags is made 
combineable).  THis can be done by supplying -Dpig.exec.nocombiner=true on the 
pig command line.

> Works in --exectype local, fails on grid - ERROR 2113: SingleTupleBag should 
> never be serialized
> ------------------------------------------------------------------------------------------------
>
>                 Key: PIG-746
>                 URL: https://issues.apache.org/jira/browse/PIG-746
>             Project: Pig
>          Issue Type: Bug
>            Reporter: David Ciemiewicz
>
> The script below works on Pig 2.0 local mode but fails when I run the same 
> program on the grid.
> I was attempting to create a workaround for PIG-710.
> Here's the error:
> {code}
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2113: 
> SingleTupleBag should never be serialized
> or serialized.
>         at org.apache.pig.data.SingleTupleBag.write(SingleTupleBag.java:129)
>         at 
> org.apache.pig.data.DataReaderWriter.writeDatum(DataReaderWriter.java:147)
>         at org.apache.pig.data.DefaultTuple.write(DefaultTuple.java:291)
>         at 
> org.apache.pig.impl.io.PigNullableWritable.write(PigNullableWritable.java:83)
>         at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
>         at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
>         at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:439)
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:101)
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:219)
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:86)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> {code}
> Here's the program:
> {code}
> A = load 'filterbug.data' using PigStorage() as ( id, str );
> A = foreach A generate
>         id,
>         str,
>         (
>         str matches 'hello' or
>         str matches 'hello'
>         ? 1 : 0
>         )                       as matched;
> describe A;
> B = group A by ( id );
> describe B;
> D = foreach B generate
>         group,
>         SUM(A.matched)  as matchedcount,
>         A;
> describe D;
> E = filter D by matchedcount > 0;
> describe E;
> F = foreach E generate
>         FLATTEN(A);
> describe F;
> dump F;
> {code}
> Here's the data filterbug.data
> {code}
> a       hello
> a       goodbye
> b       goodbye
> c       hello
> c       hello
> c       hello
> e       what
> {code}
>               

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to