[
https://issues.apache.org/jira/browse/KAFKA-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271930#comment-17271930
]
Ravishankar S R commented on KAFKA-9914:
----------------------------------------
Hi [~qq619618919],
Thanks for your reply. I did try by adding different cluster names across data
centres.
E.g: In DC A:
cluster_A_A: 10.x.x.yy:9092
cluster_A_B: 10.x.x.zz:9092
In DC B:
cluster_B_A: 10.x.x.yy:9092
cluster_B_B: 10.x.x.zz:9092
Even then heartsbeat topics were created recursively on every restart on MM.
I did workaround to overcome this by overriding upstreamTopic in custom
replicationpolicy by extending DefaultReplicationPolicy. This worked but I'm
not sure of side effects. Still testing
public String upstreamTopic(String topic)
{
String source = super.topicSource(topic);
if (source == null) \{ return null; }
else if (topic.contains(HEARTBEATS_TOPIC_NAME) && topic.contains(CLUSTER)) \{
return null; } else { return topic.substring(source.length() +
separator.length());
}
}
> Mirror Maker 2 creates heartbeats kafka topics recursively
> ----------------------------------------------------------
>
> Key: KAFKA-9914
> URL: https://issues.apache.org/jira/browse/KAFKA-9914
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 2.0.0
> Reporter: azher khan
> Priority: Major
>
> Hi Team,
> I configured Mirror Maker 2 to run on a Pod configuring the Kafka brokers in
> our environments marking '10.X.X.YY' Kafka broker as primary and '10.X.Y.XY'
> as backup.
> +*Mirror Maker mm2.properties*+
> {code:java}
> clusters=kafka,backup
> kafka.bootstrap.servers=10.X.X.YY:9092
> backup.bootstrap.servers=10.X.Y.XY:9092
> kafka->backup.enabled=true
> kafka->backup.topics=az.*
> backup->kafka.enabled=false
> sync.topic.acls.enabled=true
> {code}
>
> I was able to run Mirror Maker 2 successfully and was also able to take
> backup of the identified topics (starting with 'az').
>
> However I could see many kafka topics with the suffix 'heartbeats' created in
> recursion (See below the list of topics with 'kafka.*.heartbeats'). This
> could be because i triggered the 'connect-mirror-maker.sh' several times to
> test backing up other topics
>
> I have 2 queries
> # How to avoid having 'kafka.*.heartbeats' topic being created using Mirror
> Maker 2
> # Once Mirror Maker 2 has backed up a topic say 'azherf1test' . What is the
> best way to rollback changes made by Mirror Maker (ie delete all the topics
> created by Mirror Maker 'kafka.azherf1test' and supporting topics) ensuring
> the stability of existing/source topics and Kafka broker.
> We are testing Mirror Maker and want to ensure we are able to roll back the
> changes without affecting the Kafka topics/brokers)
> +*Kafka Topics list output:*+
> {code:java}
> azherf1test
> heartbeats
> kafka-client-topic
> mm2-configs.backup.internal
> mm2-configs.kafka.internal
> mm2-offset-syncs.backup.internal
> mm2-offsets.backup.internal
> mm2-offsets.kafka.internal
> mm2-status.backup.internal
> mm2-status.kafka.internal
> kafka.azherf1test
> kafka.checkpoints.internal
> kafka.heartbeats
> kafka.kafka.heartbeats
> kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats
> kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.kafka.heartbeats{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
