Thanks Christian. Understood, I'll try to have a look into the code and get back to you.
Le mar. 26 oct. 2021 à 19:16, Weiss, Christian <[email protected]> a écrit : > Hi Pierre, > > the Avro-Files, which we write to a Kafka Topic in the next step, needs to > be compatible with the Confluent Avro-Format, because we use Confluent > compatible deserializers on the consuming end. Therefore we need to use the > “Confluent Schema” Write-Strategy. The Hortonworks registry provides > Confluent compatible Endpoints to achieving this. > > We are actually using NiFi 11.4. > > Hope this helps. > > Thanks, > Christian > > > Christian Weiß > > Data Architect > Big Data/Analytics & IOT > > SVA System Vertrieb Alexander GmbH > Freiberger Str. 39 > 01067 Dresden > Mobile: +49 151 18027982 > > E-Mail: [email protected] > http://www.sva.de > > Geschäftsführung: Philipp Alexander, Sven Eichelbaum > Sitz der Gesellschaft: Wiesbaden > Registergericht: Amtsgericht Wiesbaden, HRB 10315 > ------------------------------ > *Von:* Pierre Villard <[email protected]> > *Gesendet:* Tuesday, October 26, 2021 7:03:36 PM > *An:* [email protected] <[email protected]> > *Betreff:* Re: Unable to evolve Schema with HWXRegistry > > Hi Christian, > > Not sure to understand the overall objective. You're writing the schema > reference using Confluent encoding but you have the schema in an > Hortonworks Schema Registry instance? > > Also, what version of NiFi are you using? > > Thanks, > Pierre > > Le mar. 26 oct. 2021 à 15:46, Weiss, Christian <[email protected]> a > écrit : > > Hey guys, > > > > we're facing some issues while writing avro with a connected hortonworks > schema registry. > > > > We would like to achieve the follwoing: > > - registered a schema and evolve it over time > > - write a avro as "confluent avro" with magic byte zero followed by the > schema id > > > > Therefor we've configured an AvroRecordSetWriter and a > HortonworksSchemaRegistry as conroller service, so that it should retrieve > the latest version. > > > > > > This works, unless we increase the version of a schema in the registry. > We've investigate the requests, responses and the avro file itself and > figured out the following. > > > > Actual behaviour: > > - The avro file contains the id which is present in the schemaMetadataId > > Expected behaviour: > > - The avro file contains the id which is present in the id field of the > response > > > > Example call: > > curl -X GET > "https://<host>:<port>/schema-registry/api/v1/schemaregistry/schemas/<schema_name>/versions/latest?branch=MASTER" > -H "accept: application/json" > > Example response: > > { > > "id": 10, > > "schemaMetadataId": 7, > > "name": "schema_name", > > "description": "test", > > "version": 1, > > "schemaText": "..", > > "timestamp": 1635252068173, > > "stateId": 5, > > "mergeInfo": null > > } > > > > This makes it impossible to increase the version of an existing schema, > because then the id and schemaMetadataId will evolve differently. > > > > Does anyone have similar issues? > > > > Thanks and cheers, > > Christian > > > > Das SVA Mail-System ist mit einem Mailverschluesselungs-Gateway > ausgestattet. Wenn Sie moechten, dass an Sie gerichtete E-Mails > verschluesselt werden, senden Sie einfach eine S/MIME-signierte E-Mail oder > Ihren PGP Public Key an [email protected]. > > The SVA mail system is equipped with an email encryption gateway. If you > want email sent to you to be encrypted please send a S/MIME signed email or > your PGP public key to [email protected]. > >
