[
https://issues.apache.org/jira/browse/KAFKA-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295425#comment-17295425
]
Luca Burgazzoli commented on KAFKA-12391:
-----------------------------------------
[~ChrisEgerton]
Option 1 is the first I took into account but since the housekeeping does not
make any sense outside the source connector, I'd prefer not to push it
downstream.
I'll have a look at option 2 which seems to be good enough for some simple
cases, however the reason I think metadata is better, is that sub-classing
would assume that the record the connector creates is also the record you
receive in the callback but - unless I've missed something - that may not be
true if a SMT during the transformation process, creates a copy of the record.
> Add an option to store arbitrary metadata to a SourceRecord
> -----------------------------------------------------------
>
> Key: KAFKA-12391
> URL: https://issues.apache.org/jira/browse/KAFKA-12391
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Luca Burgazzoli
> Priority: Minor
>
> When writing Source Connectors for Kafka, it may be required to perform some
> additional house cleaning when an record has been acknowledged by the Kafka
> broker and as today, it is possible to set up an hook by overriding
> SourceTask.commitRecord(SourceRecord).
> This works fine in most of the cases but to make it easy for the source
> connector to perform it's internal house keeping, it would be nice to have an
> option to set some additional metadata to the SourceRecord without having
> impacts to the Record sent to the Kafka Broker, something like:
> {code:java}
> class SourceRecord {
> public SourceRecord(
> ...,
> Map<String, ?> attributes) {
> ...
> this.attributes = attributes;
> }
> Map<String, ?> attributes() {
> return attributes;
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
