[
https://issues.apache.org/jira/browse/PIG-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659566#comment-13659566
]
Paul Mazak commented on PIG-2684:
---------------------------------
One simple workaround for us was to override AvroStorage's checkSchema this way.
/**
* In Pig script do:
* REGISTER 'lib/this.jar'
* DEFINE AvroStorage com.this.JoinableAvroStorage;
*/
public class JoinableAvroStorage extends AvroStorage {
@Override
public void checkSchema(ResourceSchema s) throws IOException {
try {
super.checkSchema(s);
}
catch (SchemaParseException spe) {
ResourceFieldSchema[] pigFields = s.getFields();
for (int i = 0; i < pigFields.length; i++) {
String outname = pigFields[i].getName();
if (outname.contains("::")) {
String newOutname = outname.split("::")[1];
pigFields[i].setName(newOutname);
}
}
super.checkSchema(s);
}
}
}
> :: in field name causes AvroStorage to fail
> -------------------------------------------
>
> Key: PIG-2684
> URL: https://issues.apache.org/jira/browse/PIG-2684
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Reporter: Fabian Alenius
>
> There appears to be a bug in AvroStorage which causes it to fail when there
> are field names that contain ::
> For example, the following will fail:
> data = load 'test.txt' as (one, two);
> grp = GROUP data by (one, two);
> result = foreach grp generate FLATTEN(group);
>
>
> store result into 'test.avro' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> ERROR 2999: Unexpected internal error. Illegal character in: group::one
> While the following will succeed:
> data = load 'test.txt' as (one, two);
> grp = GROUP data by (one, two);
> result = foreach grp generate FLATTEN(group) as (one,two);
>
> store result into 'test.avro' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> Here is a minimal test case:
> data = load 'test.txt' as (one::two, three);
>
>
> store data into 'test.avro' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
