[
https://issues.apache.org/jira/browse/PIG-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934569#action_12934569
]
Daniel Dai commented on PIG-1741:
---------------------------------
All tests pass. Patch is ready for review.
> Lineage fail when flatten a bag
> -------------------------------
>
> Key: PIG-1741
> URL: https://issues.apache.org/jira/browse/PIG-1741
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1741-0.patch, PIG-1741-1.patch
>
>
> The following script fail:
> {code}
> a = load '1.txt' as (a0);
> b = foreach a generate flatten((bag{tuple(map[])})a0) as b0:map[];
> c = foreach b generate (long)b0#'key1';
> dump c;
> {code}
> 1.txt:
> {([key1#1])}
> Error message:
> org.apache.pig.backend.executionengine.ExecException: ERROR 1075: Received a
> bytearray from the UDF. Cannot determine how to convert the bytearray to long.
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:286)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:361)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> In logical plan:
> {code}
> fake: Store 1-30 Schema: {long} Type: Unknown
> |
> |---c: ForEach 1-29 Schema: {long} Type: bag
> | |
> | Cast 1-28--null FieldSchema: long Type: long
> | |
> | |---MapLookup 1-26 FieldSchema: bytearray Type: bytearray
> | |
> | |---Project 1-27 Projections: [0] Overloaded: false FieldSchema:
> b0: map Type: map
> | Input: b: ForEach 1-21
> |
> |---b: ForEach 1-21 Schema: {b0: map[ ]} Type: bag
> | |
> | Cast 1-20--org.apache.pig.builtin.PigStorage FieldSchema: a0:
> bag({(map[ ])}) Type: bag
> | |
> | |---Project 1-19 Projections: [0] Overloaded: false FieldSchema:
> a0: bytearray Type: bytearray
> | Input: a: Load 1-15
> |
> |---a: Load 1-15 Schema: {a0: bytearray} Type: bag
> {code}
> Cast 1-28 get null caster. This error cannot be fixed by
> -Dusenewlogicalplan=false flag.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
