Hi Craig, I think you may be running into:
https://issues.apache.org/jira/browse/KAFKA-4073 Ismael On Thu, Oct 13, 2016 at 5:51 AM, Craig Swift < craig.sw...@returnpath.com.invalid> wrote: > Hello, > > Just to close this issue out. The 8 producer going to the 10 cluster was > the root issue. The mirror maker by default was unable to produce the > message to the destination cluster. The work around was to include a > MirrorMakerMessageHandler > that did nothing but repackage the message again. In the future it might be > nice if the mirror handled this auto-magically, but at least the ability to > alter the behavior provided an easy fix. Hope this helps someone else, > thanks. > > Craig > > Hello, > > > > I think we're misunderstanding the docs on some level and I need a little > > clarification. We have the following setup: > > > > 1) 0.8.2 producer -> writing to Kafka 0.10.0.1 cluster w/ version 10 > > message format (source cluster). > > 2) 0.10.0.1 mirror using the 'new consumer' reading from the source > > cluster and writing to Kafka 0.10.0.1 cluster w/version 0.8.2 message > > format (destination cluster). We need some of the features like SSL, > hence > > using the new consumer. > > 3) Lots of old 0.8.2 consumers reading from the destination cluster that > > still need to be upgraded. > > > > We're seeing errors from the mirror maker when trying to produce to the > > destination cluster like the following: > > > > java.lang.IllegalArgumentException: Invalid timestamp -1 > > at org.apache.kafka.clients.producer.ProducerRecord.<init> > > (ProducerRecord.java:60) > > > > Is the root problem the 0.8.2 producer sending data to the source cluster > > or the new 10 mirror writing data to the destination cluster in 0.8.2 > > format? From the docs we were under the impression that the data would be > > stored in the source cluster in 10 format regardless of the producer and > > the mirror could produce to the destination cluster regardless of it's > > message format setting. > > > > Is this current setup non-functional or is there a way to make this work? > > For example, if the mirror producing is the issue could we implement a > > custom MirrorMakerMessageHandler? Any advice and clarification would be > > helpful, thanks. > > > > Craig > > >
