[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898120#comment-16898120
]
jacob shao commented on KAFKA-7500:
-----------------------------------
Hey,
I use kubernettes to spin up 2 kafka clusters locally (3 brokers each). And
then I run MM2 locally as well to sync topic messages. When I send message to
source topic, source.topic is created in the sink cluster, but message is not
delivered. An exception is thrown in the console (see below). When I restart
MM2, message arrives in the source.topic. Anyone recognize this error?
Moreover, when I move one of the kafka cluster to a different machine,
everything works again. I tried to increase network/io threads in the local
setup, it still doesn't solve the issue.
[2019-08-01 10:18:29,033] INFO WorkerSourceTask\{id=MirrorSourceConnector-0}
flushing 21 outstanding messages for offset commit
(org.apache.kafka.connect.runtime.WorkerSourceTask:418)
[2019-08-01 10:18:29,072] INFO WorkerSourceTask\{id=MirrorSourceConnector-0}
Finished commitOffsets successfully in 39 ms
(org.apache.kafka.connect.runtime.WorkerSourceTask:500)
[2019-08-01 10:18:29,072] ERROR WorkerSourceTask\{id=MirrorSourceConnector-0}
Task threw an uncaught and unrecoverable exception
(org.apache.kafka.connect.runtime.WorkerTask:179)
{color:#FF0000}java.lang.NullPointerException{color}
{color:#FF0000} at
org.apache.kafka.connect.mirror.MirrorSourceTask.poll(MirrorSourceTask.java:140){color}
{color:#FF0000} at
org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:245){color}
{color:#FF0000} at
org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:221){color}
{color:#FF0000} at
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177){color}
{color:#FF0000} at
org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227){color}
{color:#FF0000} at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515){color}
{color:#FF0000} at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264){color}
{color:#FF0000} at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128){color}
{color:#FF0000} at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628){color}
at java.base/java.lang.Thread.run(Thread.java:834)
[2019-08-01 10:18:29,073] ERROR WorkerSourceTask\{id=MirrorSourceConnector-0}
Task is being killed and will not recover until manually restarted
(org.apache.kafka.connect.runtime.WorkerTask:180)
[2019-08-01 10:18:29,073] INFO [Producer
clientId=connector-producer-MirrorSourceConnector-0] Closing the Kafka producer
with timeoutMillis = 30000 ms.
(org.apache.kafka.clients.producer.KafkaProducer:1153)
[2019-08-01 10:18:29,080] INFO [Producer clientId=producer-7] Closing the Kafka
producer with timeoutMillis = 9223372036854775807 ms.
(org.apache.kafka.clients.producer.KafkaProducer:1153)
[2019-08-01 10:18:44,602] INFO [Worker clientId=connect-2, groupId=sec-mm2]
Attempt to heartbeat failed since coordinator localhost:31000 (id: 2147483647
rack: null) is either not started or not valid.
(org.apache.kafka.clients.consumer.internals.AbstractCoordinator:931)
[2019-08-01 10:18:44,602] INFO [Worker clientId=connect-2, groupId=sec-mm2]
Group coordinator localhost:31000 (id: 2147483647 rack: null) is unavailable or
invalid, will attempt rediscovery
(org.apache.kafka.clients.consumer.internals.AbstractCoordinator:780)
[2019-08-01 10:18:44,612] INFO [Worker clientId=connect-2, groupId=sec-mm2]
Discovered group coordinator localhost:31002 (id: 2147483645 rack: null)
(org.apache.kafka.clients.consumer.internals.AbstractCoordinator:728)
> MirrorMaker 2.0 (KIP-382)
> -------------------------
>
> Key: KAFKA-7500
> URL: https://issues.apache.org/jira/browse/KAFKA-7500
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect, mirrormaker
> Reporter: Ryanne Dolan
> Priority: Minor
> Fix For: 2.4.0
>
> Attachments: Active-Active XDCR setup.png
>
>
> Implement a drop-in replacement for MirrorMaker leveraging the Connect
> framework.
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0]
> [https://github.com/apache/kafka/pull/6295]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
