[
https://issues.apache.org/jira/browse/NIFI-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Psaltis reassigned NIFI-4380:
------------------------------------
Assignee: Andrew Psaltis
> Add Kafka Key attribute to outgoing FF for ConsumeKafkaRecord
> -------------------------------------------------------------
>
> Key: NIFI-4380
> URL: https://issues.apache.org/jira/browse/NIFI-4380
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Andrew Psaltis
> Assignee: Andrew Psaltis
>
> Often times during processing of data having access to the Key used for the
> message in Kafka is important. For example, some CDC tools that use Kafka as
> the destination for changes will use the primary key for the table as the key
> when writing data to Kafka. The value of this key becomes import after the is
> consumed from Kakfa, as many times the final destination is another data
> store, for which you need to know which column(s) of data represents the key.
> When you introduce a Schema Registry into the picture then many times the key
> is written into Kafka is often encoded, for example, if Avro is being used
> for data serialization into Kafka then the key may have its own schema that
> is serialized with. Thus, this change would need to ensure that the key is
> deserialized before being written as an attribute.
> Not sure if it gets overly complex or if it is required to have Reader/Writer
> configuration for the key as well as the value. It is certainly to be
> expected that is the key and value are stored in Kafka in Avro that the
> schema for each will be unique.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
