Dave, At one point (perhaps it is still true), in order to have a default value its type had to be the first type in the array. For optional fields, the default is null so try inverting your type arrays to match the way they're done in the input schema; namely, ["null", "string"] rather than ["string","null"]. Also if you're using a version of NiFi 1.6.0 or later, you may be able to provide a default value explicitly. Prior to NiFi 1.6.0 this was a bug [1], please see the Jira for more details (and an example).
Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-4893 On Thu, Aug 23, 2018 at 2:29 PM David Gallagher <[email protected]> wrote: > > Hi Andy – this is the schema we’re getting from the database query: > > > > "type":"record", > > "name":"view_Logons", > > "namespace":"any.data", > > "fields":[ > > {"name":"id","type":["null","int"]}, > > {"name":"timestamp","type":["null","string"]}, > > {"name":"workstation","type":["null","string"]}, > > {"name":"username","type":["null","string"]}, > > {"name":"transaction_type","type":["null","string"]}, > > {"name":"result","type":["null","string"]}, > > {"name":"message","type":["null","string"]} > > ] > > } > > > > > > And this is the schema we’re converting to: > > > > {"namespace": "com.cleverdevices.cleverreports.avro", > > "type": "record", > > "name": "AuthorizationTransaction", > > "fields": [ > > {"name": "product", "type": ["string", "null"]}, > > {"name": "sub_product", "type": ["string", "null"]}, > > {"name": "id", "type": "long"}, > > {"name": "timestamp", "type": "string"}, > > {"name": "workstation", "type": ["string", "null"]}, > > {"name": "username", "type": ["string", "null"]}, > > {"name": "transaction_type", "type": ["string"]}, > > {"name": "result", "type": "string"}, > > {"name": "message", "type": ["string", "null"]} > > ] > > } > > > > Thanks, > > > > Dave > > > > From: Andy LoPresto <[email protected]> > Sent: Thursday, August 23, 2018 1:28 PM > To: [email protected] > Subject: Re: ConvertAvroSchema with unmapped fields > > > > Hi Dave, > > > > Can you provide the two schemas (redact anything necessary). There is a way > to specify an “optional” field [1] by setting the type to an array of null > and the type you support. You can also specify a default value if you wish, > which will be set for records that do not contain a value there: > > > > { > > "type": "record", > > "name": "Address", > > "fields" : [ > > {"name": "streetNumber", "type": "int"}, > > {"name": "aptNumber", "type": ["null", "int"]}, // optional apt number > > {"name": "country", "type": "string", "default": "US"}, // default country > > ] > > } > > > > > > [1] https://avro.apache.org/docs/1.8.1/spec.html#schema_record > > > > > > Andy LoPresto > > [email protected] > > [email protected] > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > On Aug 23, 2018, at 10:07 AM, David Gallagher <[email protected]> > wrote: > > > > Hi – I’ve got a scenario where I’m trying to convert the implicit AVRO schema > associated with a database record to a different AVRO schema. I’m trying to > use the ConvertAvroSchema processor, but it won’t validate because there is > an ‘unmapped’ field that exists in the second schema but not the first. Now, > I could work around this issue by including a constant in the database > record, but I would prefer to have a null or default value supplied by the > second schema instead. Is there some setting I’m missing that would let me > specify the behavior for unmapped fields? > > > > Thanks, > > > > Dave > >
