[
https://issues.apache.org/jira/browse/NIFI-7376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229094#comment-17229094
]
Alasdair Brown edited comment on NIFI-7376 at 11/10/20, 9:53 AM:
-----------------------------------------------------------------
FYI see NIFI-7917 which is tracking Avro upgrade to 1.10
was (Author: sdairs):
FYI see NIFI-7917
> Avro Single-object encoding Support
> -----------------------------------
>
> Key: NIFI-7376
> URL: https://issues.apache.org/jira/browse/NIFI-7376
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.9.2
> Reporter: Nathan English
> Priority: Minor
>
> For my Flows I consume Avro Binary Encoded Messages from Kafka which is
> currently working great! However going forward one of our inputs is looking
> to provide [Avro Single-object
> Encoding|[https://avro.apache.org/docs/1.8.2/spec.html#single_object_encoding]].
>
> Avro Single Object Encoding provides us a way to where we don't have to have
> the overhead and message size increase of embedding the schema in the
> message. Single Object Encoding achieves this with a couple of extra fields
> to firstly confirm the message is an Avro message and secondly a fingerprint
> of the schema used to encode the message.
> This is a massive benefit for us, because we have multiple of the same device
> producing messages into one Kafka Topic. This is fine until we start
> upgrading these devices where schema changes may occur, this is when schema
> fingerprinting comes into its element.
> From all the information I have found, it looks to me as if Avro Single
> Object Encoding was added in version 1.8.2 of the Avro Specification. NiFi is
> currently using version 1.8.1 of the Avro Specification based on the
> [pom|[https://github.com/apache/nifi/blob/rel/nifi-1.11.4/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/pom.xml]]
> in NiFi Record Serialization Services section of the github project.
> I'm sure there are a million ways to tackle this issue and I'm personally
> haven't done enough research on NiFi or Avro on it to suggest a way to
> resolve this, but I can tell it's not as simple as just upgrading the Avro
> version used.
> My thoughts were to upgrade the Library version, then modify the avro schema
> registry to either take in a fingerprint value or have it calculated on
> enablement of the registry. I'm sure it's probably not as simple as I have
> just made it!.
> More than happy to help where I can.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
