[
https://issues.apache.org/jira/browse/PIG-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269190#comment-13269190
]
Fabian Alenius commented on PIG-2684:
-------------------------------------
This appears to be limitation in avro itself:
[http://avro.apache.org/docs/current/spec.html#Names]
The name portion of a fullname, record field names, and enum symbols must:
* start with [A-Za-z_]
* subsequently contain only [A-Za-z0-9_]
The only semi-reasonable solution I can see to this problem is to encode the
prefix in the namespace.
In record, enum and fixed definitions, the fullname is determined in one of the
following ways:
* A name and namespace are both specified. For example, one might use "name":
"X", "namespace": "org.foo" to indicate the fullname org.foo.X.
Anyone have some better idea?
> :: in field name causes AvroStorage to fail
> -------------------------------------------
>
> Key: PIG-2684
> URL: https://issues.apache.org/jira/browse/PIG-2684
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Reporter: Fabian Alenius
>
> There appears to be a bug in AvroStorage which causes it to fail when there
> are field names that contain ::
> For example, the following will fail:
> data = load 'test.txt' as (one, two);
> grp = GROUP data by (one, two);
> result = foreach grp generate FLATTEN(group);
>
>
> store result into 'test.avro' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> ERROR 2999: Unexpected internal error. Illegal character in: group::one
> While the following will succeed:
> data = load 'test.txt' as (one, two);
> grp = GROUP data by (one, two);
> result = foreach grp generate FLATTEN(group) as (one,two);
>
> store result into 'test.avro' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> Here is a minimal test case:
> data = load 'test.txt' as (one::two, three);
>
>
> store data into 'test.avro' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
