Hi Andy – this is the schema we’re getting from the database query:
"type":"record",
"name":"view_Logons",
"namespace":"any.data",
"fields":[
{"name":"id","type":["null","int"]},
{"name":"timestamp","type":["null","string"]},
{"name":"workstation","type":["null","string"]},
{"name":"username","type":["null","string"]},
{"name":"transaction_type","type":["null","string"]},
{"name":"result","type":["null","string"]},
{"name":"message","type":["null","string"]}
]
}
And this is the schema we’re converting to:
{"namespace": "com.cleverdevices.cleverreports.avro",
"type": "record",
"name": "AuthorizationTransaction",
"fields": [
{"name": "product", "type": ["string", "null"]},
{"name": "sub_product", "type": ["string", "null"]},
{"name": "id", "type": "long"},
{"name": "timestamp", "type": "string"},
{"name": "workstation", "type": ["string", "null"]},
{"name": "username", "type": ["string", "null"]},
{"name": "transaction_type", "type": ["string"]},
{"name": "result", "type": "string"},
{"name": "message", "type": ["string", "null"]}
]
}
Thanks,
Dave
From: Andy LoPresto <[email protected]>
Sent: Thursday, August 23, 2018 1:28 PM
To: [email protected]
Subject: Re: ConvertAvroSchema with unmapped fields
Hi Dave,
Can you provide the two schemas (redact anything necessary). There is a way to
specify an “optional” field [1] by setting the type to an array of null and the
type you support. You can also specify a default value if you wish, which will
be set for records that do not contain a value there:
{
"type": "record",
"name": "Address",
"fields" : [
{"name": "streetNumber", "type": "int"},
{"name": "aptNumber", "type": ["null", "int"]}, // optional apt number
{"name": "country", "type": "string", "default": "US"}, // default country
]
}
[1] https://avro.apache.org/docs/1.8.1/spec.html#schema_record
Andy LoPresto
[email protected]<mailto:[email protected]>
[email protected]<mailto:[email protected]>
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
On Aug 23, 2018, at 10:07 AM, David Gallagher
<[email protected]<mailto:[email protected]>> wrote:
Hi – I’ve got a scenario where I’m trying to convert the implicit AVRO schema
associated with a database record to a different AVRO schema. I’m trying to use
the ConvertAvroSchema processor, but it won’t validate because there is an
‘unmapped’ field that exists in the second schema but not the first. Now, I
could work around this issue by including a constant in the database record,
but I would prefer to have a null or default value supplied by the second
schema instead. Is there some setting I’m missing that would let me specify the
behavior for unmapped fields?
Thanks,
Dave
