[
https://issues.apache.org/jira/browse/KAFKA-17784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889019#comment-17889019
]
Greg Harris commented on KAFKA-17784:
-------------------------------------
[~yitian998] Here's the error causing the shutdown:
{noformat}
[2024-10-13 09:05:12,622] ERROR [Worker clientId=idcb->idca, groupId=idcb-mm2]
Uncaught exception in herder work thread, exiting:
(org.apache.kafka.connect.runtime.distributed.DistributedHerder:389)
org.apache.kafka.common.config.ConfigException: Topic
'mm2-offsets.idcb.internal' supplied via the 'offset.storage.topic' property is
required to have 'cleanup.policy=compact' to guarantee consistency and
durability of source connector offsets, but found the topic currently has
'cleanup.policy=delete'. Continuing would likely result in eventually losing
source connector offsets and problems restarting this Connect cluster in the
future. Change the 'offset.storage.topic' property in the Connect worker
configurations to use a topic with 'cleanup.policy=compact'.
at
org.apache.kafka.connect.util.TopicAdmin.verifyTopicCleanupPolicyOnlyCompact(TopicAdmin.java:581)
at
org.apache.kafka.connect.storage.KafkaTopicBasedBackingStore.lambda$topicInitializer$0(KafkaTopicBasedBackingStore.java:47)
at
org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:247)
at
org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:231)
at
org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:242)
at org.apache.kafka.connect.runtime.Worker.start(Worker.java:233)
at
org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:163)
at
org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:373)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.base/java.lang.Thread.run(Unknown Source){noformat}
> Mirror Maker2 Pod CrashLoopbackOff
> ----------------------------------
>
> Key: KAFKA-17784
> URL: https://issues.apache.org/jira/browse/KAFKA-17784
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 3.7.1
> Reporter: George Yang
> Priority: Blocker
> Attachments: connect-1.log
>
>
> When I use k8s deployment with mirrormaker v3.7.1, and deploy one kafka node
> in each data center, I always got the crashloopbackoff error, please see the
> attachment.
>
> The configuration of mirrormaker is:
> ```configuration
> clusters = idca, idcb
> idca.bootstrap.servers = 192.168.2.146:13399
> idcb.bootstrap.servers = 192.168.2.147:13399
> idca->idcb.enabled = true
> idca->idcb.topics = .*
> idcb->idca.enabled = true
> idcb->idca.topics = .*
> replication.factor=1
> tasks.max=6
> emit.checkpoints.interval.seconds=5
> idca.producer.acks=all
> idca.producer.batch.size=50000
> idca.consumer.auto.offset.reset=latest
> idcb.consumer.auto.offset.reset=latest
> idca.consumer.max.poll.interval.ms=20000
> idcb.consumer.max.poll.interval.ms=20000
> refresh.topics.enabled=true
> refresh.topics.interval.seconds=5
> refresh.groups.enabled=true
> refresh.groups.interval.seconds=5
> dedicated.mode.enable.internal.rest = true
> idca.scheduled.rebalance.max.delay.ms=20000
> idcb.scheduled.rebalance.max.delay.ms=20000
> checkpoints.topic.replication.factor=3
> heartbeats.topic.replication.factor=3
> offset-syncs.topic.replication.factor=3
> offset.storage.replication.factor=3
> status.storage.replication.factor=3
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
