Hello -- Avro supports a map type:
https://avro.apache.org/docs/1.9.0/spec.html#Maps
Generating an Avro schema from a JSON example can be ambiguous since a
JSON object can either be converted to a record or a map. You're
probably looking for something like this:
{
"type" : "record",
"name" : "MyClass",
"namespace" : "com.acme.avro",
"fields" : [ {
"name" : "one_level",
"type" : {
"type" : "record",
"name" : "one_level",
"fields" : [ {
"name" : "inner_level",
"type" : {
"type" : "map",
"values" : {
"type" : "record",
"name" : "sample",
"fields" : [ {
"name" : "sample1",
"type" : "string"
}, {
"name" : "sample2",
"type" : "string"
} ]
}
}
} ]
}
} ]
}
On Tue, Aug 6, 2019 at 10:47 AM Edgar H <[email protected]> wrote:
>
> I'm trying to translate a schema that I have in Spark which is defined for
> Parquet, and I would like to use it within Avro too.
>
> StructField("one_level", StructType(List(StructField(
> "inner_level",
> MapType(
> StringType,
> StructType(
> List(
> StructField("field1", StringType),
> StructField("field2", ArrayType(StringType))
> )
> )
> )
> )
> )), nullable = false)
>
> However, in Avro I haven't seen any examples of Maps containing Record type
> objects...
>
> Tried a sample input with an online Avro schema generator, taking this input.
>
> {
> "one_level": {
> "inner_level": {
> "sample1": {
> "field1": "sample",
> "field2": ["a", "b"],
> },
> "sample2": {
> "field1": "sample2",
> "field2": ["a", "b"]
> }
> }
> }
>
> }
>
> It prompts this output.
>
> {
> "name": "MyClass",
> "type": "record",
> "namespace": "com.acme.avro",
> "fields": [
> {
> "name": "one_level",
> "type": {
> "name": "one_level",
> "type": "record",
> "fields": [
> {
> "name": "inner_level",
> "type": {
> "name": "inner_level",
> "type": "record",
> "fields": [
> {
> "name": "sample1",
> "type": {
> "name": "sample1",
> "type": "record",
> "fields": [
> {
> "name": "field1",
> "type": "string"
> },
> {
> "name": "field2",
> "type": {
> "type": "array",
> "items": "string"
> }
> }
> ]
> }
> },
> {
> "name": "sample2",
> "type": {
> "name": "sample2",
> "type": "record",
> "fields": [
> {
> "name": "field1",
> "type": "string"
> },
> {
> "name": "field2",
> "type": {
> "type": "array",
> "items": "string"
> }
> }
> ]
> }
> }
> ]
> }
> }
> ]
> }
> }
> ]
> }
>
> Which isn't absolutely what I'm looking for. Is it possible to define such
> schema in Avro?
