[ https://issues.apache.org/jira/browse/KAFKA-7629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Darshan Mehta updated KAFKA-7629: --------------------------------- Description: *Setup:* I have 2 kafka images running Spoify Kafka image [https://hub.docker.com/r/spotify/kafka] Config: Image 1: * host: kafka1 * zk port : 2181 * broker port : 9092 Image 2: * host: kafka2 * zk port : 1181 * broker port : 8092 Producer Properties for Mirror maker: {code:java} bootstrap.servers=kafka2:8092 {code} Consumer Properties for Mirror maker: {code:java} bootstrap.servers=kafka1:9092 group.id=test-consumer-group exclude.internal.topics=true {code} *Steps to replicate :* # Start mirror maker with following command : {code:java} $KAFKA_INSTALLATION_DIR/bin/kafka-mirror-maker.sh --producer.config <producer_config_location> --consumer.config <consumer_config_location> --num.streams 1 --whitelist topic-1 {code} # Start local kafka console consumer to listen to topic-1 for kafka2:8092 {code:java} $KAFKA_INSTALLATION_DIR/bin/kafka-console-consumer.sh --bootstrap-server kafka2:8092 --topic topic-1 {code} # Produce an event to kafka1:9092 - topic-1 -> It will be printed on the console in Step 2 # Stop mirror maker with ctrl+C (started in step 1) # Restart mirror maker with same command # Produce an event onto the same topic (i.e. repeat step 3) # Both source and destination will be flooded with the same messages until mirror maker is stopped Surprisingly, source kafka also gets flooded with the same message. I believe when restarted, the mirror maker is unable to read the state? was: *Setup:* I have 2 kafka images running Spoify Kafka image : [https://hub.docker.com/r/spotify/kafka/|https://hub.docker.com/r/spotify/kafka/)] Config: Image 1: * host: kafka1 * zk port : 2181 * broker port : 9092 Image 2: * host: kafka2 * zk port : 1181 * broker port : 8092 Producer Properties for Mirror maker: {code:java} bootstrap.servers=kafka2:8092 {code} Consumer Properties for Mirror maker: {code:java} bootstrap.servers=kafka1:9092 group.id=test-consumer-group exclude.internal.topics=true {code} *Steps to replicate :* # Start mirror maker with following command : {code:java} $KAFKA_INSTALLATION_DIR/bin/kafka-mirror-maker.sh --producer.config <producer_config_location> --consumer.config <consumer_config_location> --num.streams 1 --whitelist topic-1 {code} # Start local kafka console consumer to listen to topic-1 for kafka2:8092 {code:java} $KAFKA_INSTALLATION_DIR/bin/kafka-console-consumer.sh --bootstrap-server kafka2:8092 --topic topic-1 {code} # Produce an event to kafka1:9092 - topic-1 -> It will be printed on the console in Step 2 # Stop mirror maker with ctrl+C (started in step 1) # Restart mirror maker with same command # Produce an event onto the same topic (i.e. repeat step 3) # Both source and destination will be flooded with the same messages until mirror maker is stopped Surprisingly, source kafka also gets flooded with the same message. I believe when restarted, the mirror maker is unable to read the state? > Mirror maker goes into infinite loop > ------------------------------------ > > Key: KAFKA-7629 > URL: https://issues.apache.org/jira/browse/KAFKA-7629 > Project: Kafka > Issue Type: Bug > Components: mirrormaker > Affects Versions: 2.0.0 > Environment: local > Reporter: Darshan Mehta > Priority: Major > > *Setup:* > I have 2 kafka images running Spoify Kafka image > [https://hub.docker.com/r/spotify/kafka] > Config: > Image 1: > * host: kafka1 > * zk port : 2181 > * broker port : 9092 > Image 2: > * host: kafka2 > * zk port : 1181 > * broker port : 8092 > Producer Properties for Mirror maker: > {code:java} > bootstrap.servers=kafka2:8092 > {code} > Consumer Properties for Mirror maker: > {code:java} > bootstrap.servers=kafka1:9092 > group.id=test-consumer-group > exclude.internal.topics=true > {code} > > *Steps to replicate :* > # Start mirror maker with following command : > {code:java} > $KAFKA_INSTALLATION_DIR/bin/kafka-mirror-maker.sh --producer.config > <producer_config_location> --consumer.config <consumer_config_location> > --num.streams 1 --whitelist topic-1 > {code} > # Start local kafka console consumer to listen to topic-1 for kafka2:8092 > {code:java} > $KAFKA_INSTALLATION_DIR/bin/kafka-console-consumer.sh --bootstrap-server > kafka2:8092 --topic topic-1 > {code} > # Produce an event to kafka1:9092 - topic-1 -> It will be printed on the > console in Step 2 > # Stop mirror maker with ctrl+C (started in step 1) > # Restart mirror maker with same command > # Produce an event onto the same topic (i.e. repeat step 3) > # Both source and destination will be flooded with the same messages until > mirror maker is stopped > Surprisingly, source kafka also gets flooded with the same message. I believe > when restarted, the mirror maker is unable to read the state? -- This message was sent by Atlassian JIRA (v7.6.3#76005)