Josh Highley created NIFI-9250: ---------------------------------- Summary: PublishKafka, PublishKafkaRecord: send tombstone messages Key: NIFI-9250 URL: https://issues.apache.org/jira/browse/NIFI-9250 Project: Apache NiFi Issue Type: Improvement Reporter: Josh Highley
The Kafka Connect JDBC Sink Connector supports database deletes using what's called tombstone messages. The Kafka message key contains the primary key of the database record to be deleted and the message value is NULL. The Publish Kafka processors do not appear to support the necessary formats for the key or value: 1) There's not a way to send a null message value. 2) If the DB key is a single column, the message key can just be the value. However, if the primary key is a composite key (more than 1 column) then the message key needs a schema. This could be JSON with embedded schema but this is inefficient. Ideally, I'd be able to write an Avro message with a schema registry reference to the Kafka message key. -- This message was sent by Atlassian Jira (v8.3.4#803005)