[
https://issues.apache.org/jira/browse/PIG-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880376#comment-13880376
]
Rohini Palaniswamy commented on PIG-3661:
-----------------------------------------
This patch fixes other issues with AvroStorage as well apart from fixing
multiple load and store
- Hidden files were not excluded (PIG-3717)
- mapred.input.dir was getting populated with all files instead of the top
level directory making the conf very big
- Default value was not set for a Union
https://reviews.apache.org/r/17266/
> Piggybank AvroStorage fails if used in more than one load or store statement
> ----------------------------------------------------------------------------
>
> Key: PIG-3661
> URL: https://issues.apache.org/jira/browse/PIG-3661
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11.1
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.13.0
>
> Attachments: PIG-3661-1.patch
>
>
> To reproduce:
> A =load '/tmp/data' as (a1:int, a2:int, a3:int);
> B = load '/tmp/data1' as (b1:chararray, b2:chararray, b3:chararray);
> store A into '/tmp/out/a' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> store B into '/tmp/out2/b' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> It either fails in the map job if schema is incompatible, or B gets schema of
> A and B merged leading to incorrect results.
> Reason is schema is stored and accessed from a property of UDFContext without
> using a context signature.
> UDFContext context = UDFContext.getUDFContext();
> Properties property = context.getUDFProperties(ResourceSchema.class);
> String prevSchemaStr =
> property.getProperty(AVRO_OUTPUT_SCHEMA_PROPERTY);
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
