Hello,
I am currently using Kafka mirror maker v2 v3.2.0 to replicate a Kafka
cluster version 3.2.0 to another Kafka cluster version 3.2.0. However, I am
experiencing a delay issue where it takes around 10 minutes for a topic to
be created in the destination cluster after it has been created in the
source cluster. I am replicating all topics and groups with identity
replication and have provided the configs of the mirrors below.
When I check the status, all the tasks are running smoothly and the logs
show no errors (I even increased the connect.root.logger.level to DEBUG).
The CPU and memory consumption of the cluster are both below the request
levels.
Despite everything appearing to be fine, I am still experiencing a delay
when I create a topic in the source cluster. I should note that once the
topic is created, message replication is instantaneous, it is only the
topic creation that is delayed by a few minutes.
Here are the configs for the different connectors of the MM2 cluster:
mirrors:
- sourceConnector:
config:
consumer.auto.offset.reset: latest
sync.topic.acls.enabled: "false"
replication.policy.separator: ""
replication.policy.class:
org.apache.kafka.connect.mirror.IdentityReplicationPolicy
heartbeatConnector:
config:
heartbeats.topic.replication.factor: 1
checkpointConnector:
tasksMax: 6
config:
replication.policy.separator: ""
replication.policy.class:
org.apache.kafka.connect.mirror.IdentityReplicationPolicy
checkpoints.topic.replication.factor: 1
sync.group.offsets.enabled: "true"
topicsPattern: ".*"
groupsPattern: ".*"
Thank you for your help on this.
