[
https://issues.apache.org/jira/browse/PIG-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Kamath updated PIG-464:
-------------------------------
Status: Patch Available (was: Open)
Attached patch with unit test. The change was to use canonical name of
FieldSchema as the key in the mFieldSchemas multimap. Earlier the FieldSchema
object was the key and this caused problems when the the FieldSchema object was
changed (say the schema or some other member was changed) while in the
multimap. This caused the hashcode of the FieldSchema object to change and
hence corrupt the multimap.
> Loading data with a schema that has a bag as a field results in error
> ---------------------------------------------------------------------
>
> Key: PIG-464
> URL: https://issues.apache.org/jira/browse/PIG-464
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: types_branch
>
> Attachments: PIG-464.patch
>
>
> Script:
> {code}
> a = load '/user/pig/tests/data/singlefile/studenttab10k' using PigStorage()
> as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3), ['key1'#'value1','key2'#'value2'];
> -- store the bag, tuple and map
> store c into '/tmp/intermediate' using BinStorage();
> d = load '/tmp/intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)},
> t2:tuple(a,b,c), m:map[]);
> e = foreach d generate COUNT(b), t2.a, t2.b, t2.c, m#'key1', m#'key2';
> dump e;
> {code}
> Error:
> {noformat}
> java.io.IOException: Unable to open iterator for alias: e [Unable to store
> for alias: e [Unable to insert type casts into plan]]
> at
> org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:150)
> at
> org.apache.pig.impl.plan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:65)
> at org.apache.pig.PigServer.compileLp(PigServer.java:723)
> at org.apache.pig.PigServer.compileLp(PigServer.java:655)
> at org.apache.pig.PigServer.store(PigServer.java:433)
> at org.apache.pig.PigServer.store(PigServer.java:421)
> at org.apache.pig.PigServer.openIterator(PigServer.java:384)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
> at org.apache.pig.Main.main(Main.java:306)
> Caused by: java.io.IOException: Unable to store for alias: e [Unable to
> insert type casts into plan]
> ... 12 more
> Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: Unable to
> insert type casts into plan
> ... 12 more
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.<init>(ArrayList.java:133)
> at org.apache.pig.impl.util.MultiMap.put(MultiMap.java:82)
> at
> org.apache.pig.impl.logicalLayer.schema.Schema.clone(Schema.java:725)
> at
> org.apache.pig.impl.logicalLayer.schema.Schema$FieldSchema.clone(Schema.java:348)
> at
> org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:131)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
