Hi Ryanne,
Thanks for responding!
I am using only one node to run the mm atm. I found some luck in enabling the
connect.log and now it is printing a lot of lines like below -
[2021-10-25 20:13:24,477] WARN [Worker clientId=connect-2, groupId=A-mm2] 51
partitions have leader brokers without a matching listener, including
[__consumer_offsets-0, __consumer_offsets-10, __consumer_offsets-20,
__consumer_offsets-40, __consumer_offsets-30, __consumer_offsets-9,
__consumer_offsets-39, __consumer_offsets-11, __consumer_offsets-31,
__consumer_offsets-13] (org.apache.kafka.clients.NetworkClient:1070)[2021-10-25
20:13:24,512] INFO AbstractConfig values:
(org.apache.kafka.common.config.AbstractConfig:347)[2021-10-25 20:13:24,515]
INFO Connector B->A configured.
(org.apache.kafka.connect.mirror.MirrorMaker:212)org.apache.kafka.connect.runtime.rest.errors.BadRequestException:
Connector configuration is invalid and contains the following 1
error(s):Invalid value
io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy for configuration
replication.policy.class: Class
io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy could not be found.
but mostly this one - [2021-10-25 20:23:48,430] WARN [Worker
clientId=connect-2, groupId=A-mm2] 51 partitions have leader brokers without a
matching listener, including [__consumer_offsets-0, __consumer_offsets-10,
__consumer_offsets-20, __consumer_offsets-40, __consumer_offsets-30,
__consumer_offsets-9, __consumer_offsets-39, __consumer_offsets-11,
__consumer_offsets-31, __consumer_offsets-13]
(org.apache.kafka.clients.NetworkClient:1070)
Could you help me understand the WARN. Now I have stopped it as this looked
confusing [2021-10-25 20:13:24,515] INFO Connector B->A configured.
(org.apache.kafka.connect.mirror.MirrorMaker:212) as my data is on A which
needs to be mirrored to B and that is what I have set in my mm2.properties
(pasting it again fyr)
A->B.enabled = true
B->A.enabled = false
Thanks again for your help!
~
On Tuesday, 26 October, 2021, 01:46:54 am IST, Ryanne Dolan
<ryannedo...@gmail.com> wrote:
Hello, how many mm nodes are you using? Try starting with one and adding
more after the first starts working. There is a known race that effects
some people trying to start multiple nodes at once.
Also check for any potential auth problems. The driver isn't very verbose
when it comes to auth problems, and it's possible for some nodes to have
the right certs while others don't (due to the distributed nature of
Connect). For example, I've often seen the leader node have the proper
configuration while the other Workers don't have the same creds in the same
place.
Ryanne
On Mon, Oct 25, 2021, 2:42 PM Rijo Roy <rjo_...@yahoo.com.invalid> wrote:
> Hi,
>
> I was very much relieved to see MirrorMaker2 working in my sandbox for a
> major project where we are migrating out of cluster A to cluster B for a
> better hardware. To my surprise, it fails to migrate any user topics when I
> used it in one of our real pre-prod environments where we have real topics.
> Let me give a brief about my environment -
>
> ENV A
>
> OS: Ubuntu 18
> 3 node Kafka cluster
> 3 node ZooKeeper cluster
>
> ENV B
> OS: Ubuntu 18
> 3 node Kafka cluster
> 3 node ZooKeeper cluster
>
> Kafka version: Confluent kafka 5.5
> Zk version: 3.5.8
>
> here is the mm2.properties I used in my sandbox fyr where replication is
> working fine -
> # mm2.propertiesclusters = A, BA.bootstrap.servers = Ahost1:port,
> Ahost2:port, Ahost3:portB.bootstrap.servers =
> Bhost1:port, Bhost2:port, Bhost3:portA->B.enabled = trueB->A.enabled =
> falsereplication.factor=3checkpoints.topic.replication.factor=3heartbeats.topic.replication.factor=3offset-syncs.topic.replication.factor=3replication.policy.separator=source.cluster.alias=target.cluster.alias=replication.policy.class=
> io.strimzi.kafka.connect.mirror.IdentityReplicationPolicytopics = .*groups
> = .*emit.checkpoints.interval.seconds =
> 10security.protocol=SASL_PLAINTEXTsasl.mechanism=PLAINsasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
> required \ username="vvvvv" \ password="ccccccccc";
>
> I used the same configuration in my pre-prod environment where I have to
> successfully test the cutover procedure inorder to move to PROD within the
> speculated tight timeline but it fails to replicate any data. The only job
> it is successfully doing is creating these 3 below topics in both A and B
> clusters -
> mm2-configs.n.internalmm2-offsets.n.internalmm2-status.n.internal
> where n is A in source and B in target
> And the kafka logs does not have anything to help me with troubleshooting
> neither have any monitoring tools to track the sync progress if at all it
> is happening.
>
> Could you please help me here -
> 1. How to enable logging for mirrormaker
> 2. How can I successfully monitor it
> 3. How can I make it work
>
> Note: Its not a network traffic issue as I have verified telnet from
> A<-->B and it is working.
>
> Thanks!
>
