Gwen, thanks for the reply: Just to confirm - are we talking about the combination of the IDs that I layed out as being the "CorrelationID" ?
I guess the alternative is to pass our own CorrelationID as part of the data. Would you agree with this? I do see something at this address that talks about a "user defined" CorrelationID https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ProduceResponse CorrelationId: This is a user-supplied integer. It will be passed back in the response by the server, unmodified. It is useful for matching request and response between the client and server. -----Original Message----- From: Gwen Shapira [mailto:[email protected]] Sent: Wednesday, December 09, 2015 12:50 PM To: [email protected] Subject: Re: Match Producer and RecordMetadata with Consumer and ConsumerRecord Correlation ID is for a request (i.e. separate ID for produce request and a fetch request), not a record. So it can't be used in the way you are trying to. On Wed, Dec 9, 2015 at 9:30 AM, John Menke <[email protected]> wrote: > Can a correlationID be created from a ConsumerRecord that will allow > for identification of the corresponding RecordMetaData instance that > was returned from the Producer.send() method? > > I am Looking at the JavaDocs and the Producer returns RecordMetadata > which has the following signature: > > RecordMetadata(TopicPartition topicPartition, long baseOffset, long > relativeOffset) > > I am not sure if this can match to > > ConsumerRecord(java.lang.String topic, int partition, long offset, K > key, V value) > > Can we match producer requests to Consumer Records by matching these > values? > > RecordMetaData.TopicPartion.topic = ConsumerRecord.topic > RecordMetaData.TopicPartion.partition = ConsumerRecord.partition > RecordMetaData(baseOffset + relativeOffset) = ConsumerRecord.offset > > In other words, can a CorrelationID be created from these values that > will allow Consumers to link back to the Producer send() event. (In > the client calling code) > > > > > > >
