[
https://issues.apache.org/jira/browse/PIG-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350543#comment-14350543
]
Ratandeep Ratti commented on PIG-4448:
--------------------------------------
For instance, below is the Avro schema and Pig schema for an Avro record which
contains a map which contain a record.
{code}
Avro schema
{
"type": "record",
"name": "recordInMap",
"namespace": "org.apache.pig.test.builtin",
"fields": [
{
"name": "key",
"type": "string"
},
{
"name": "value",
"type": "int"
},
{
"name": "parameters",
"type": {
"type": "map",
"values": {
"type": "record",
"name": "A",
"fields": [
{
"name": "id",
"type": [
"null",
"int"
]
}
]
}
}
}
]
}
Pig schema
key:chararray,value:int,parameters:[A:(id:int)]
{code}
According to this, the Pig representation should contain a tuple for the the
map's value record, but instead it currently contains an Avro GenericRecord
Updating the patch. Now I'm using the logic from
org.apache.pig.impl.util.avro.AvroBagWrapper on how it is converting each
element of the Bag to an equivalent Pig type.
My only gripe now is that the name of the method which helps to convert types
from avro to Pig in AvroTupleWrapper is "resolveUnion" . Ideally I'd love to
name it avroToPig or something similar but since its scope is public I'm
refraining to do so.
> AvroMapWrapper leaks Avro data types when the map values are complex avro
> records
> ---------------------------------------------------------------------------------
>
> Key: PIG-4448
> URL: https://issues.apache.org/jira/browse/PIG-4448
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.13.0
> Reporter: Ratandeep Ratti
> Assignee: Ratandeep Ratti
> Fix For: 0.15.0
>
> Attachments: avro_map_wrapper_fix.patch
>
>
> AvroMapWrapper does not convert value objects in it's inner map to Pig types
> when queried. This could lead to problems when the value object is a complex
> Avro type.
> Attaching a preliminary patch. Please let me know if this is not the best way
> to fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
