Re: Map with another map inside (unpredictable naming)

[Dag Stockstad]

Thank you so much for this. This works perfectly. I tried simply putting
the string "map" as one of the values but that obviously didn't work. I
think this would be worth calling out in documentation. Happy to submit PR.
Is that possible?

Thanks again. This really helped me.

Kind regards,
Dag

On Mon, Mar 27, 2017 at 3:20 PM, Harsh J <ha...@cloudera.com> wrote:

> The map union value you currently have can certainly carry another map
> type within. Here's how you'd probably want to define it:
>
> {
>     "name": "metadata",
>     "type": {
>         "type": "map",
>         "values": [
>             "null",
>             "int",
>             "float",
>             "string",
>             "boolean",
>             "long",
>             {
>                 "type": "map",
>                 "values": [
>                     "null",
>                     "int",
>                     "float",
>                     "string",
>                     "boolean",
>                     "long"
>                 ]
>             }
>         ]
>     }
> }
>
> P.s. Its generally better to specify "null" as the first union type:
> http://avro.apache.org/docs/current/spec.html#Unions
>
> On Mon, 27 Mar 2017 at 16:47 Dag Stockstad <d...@intercom.io> wrote:
>
>> Hi Avro aficionados,
>>
>> I'm having trouble serializing a record with a nested map structure i.e.
>> a map within a map. The record I'm trying to send has the following
>> structure:
>> {
>>     "event_type": "some_type",
>>     "data": {
>>         "id": "2f720f90-ea06-4248-a72e-01eea44981ed",
>>         "metadata": {
>>             "some_attr": "some_value",
>>             "some_map_with_unpredictable_name": {
>>                 "some_attr": "some_value"
>>             }
>>         }
>>     }
>> }
>>
>> And the schema is this:
>> {
>>     "namespace": "org.example.event.avro",
>>     "type": "record",
>>     "name": "EventNotification",
>>     "fields": [{
>>         "name": "event_type",
>>         "type": "string"
>>     }, {
>>         "name": "data",
>>         "type": {
>>             "type": "record",
>>             "name": "EventData",
>>             "fields": [{
>>                 "name": "id",
>>                 "type": "string"
>>             }, {
>>                 "name": "metadata",
>>                 "type": {
>>                     "type": "map",
>>                     "values": [
>>                         "int",
>>                         "float",
>>                         "string",
>>                         "boolean",
>>                         "long",
>>                         "null"
>>                     ]
>>                 }
>>             }]
>>         }
>>     }]
>> }
>>
>> The nested map (some_map_with_unpredictable_name) is causing problems
>> (serialization error). Is there any way I can have another map as a value
>> in the metadata map?
>>
>> Due to the nature of the system, I cannot 100% predict the structure of
>> the metadata field. Can Avro accomodate these requirements or do I have to
>> fall back on something such as JSON for this one?
>>
>> Help very appreciated (I'm a bit stuck).
>>
>> Kind regards,
>> Dag
>>
>>