[
https://issues.apache.org/jira/browse/KAFKA-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhenChun Pan updated KAFKA-13465:
---------------------------------
Description:
Hi Team
Mirror Maker: 2.7.0
when i enable auto create topic in both side:
auto.create.topics.enable=true
sometimes，mirror maker inner topic create by server not expected，mirror naker
start error.
```
[2021-11-19 18:03:56,707] ERROR [Worker clientId=connect-2, groupId=pr-mm2]
Uncaught exception in herder work thread, exiting:
(org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.common.config.ConfigException: Topic 'mm2-configs.pr.internal'
supplied via the 'config.storage.topic' property is required to have
'cleanup.policy=compact' to guarantee consistenc****{*}#{*}#*****connector
configurations, but found the topic currently has 'cleanup.policy=delete'.
Continuing would likely result in eventually losing connector configurations
and problems restarting this Connect cluster in the future. Change the
'config.storage.topic' property in the Connect worker configurations to use a
topic with 'cleanup.policy=compact'.
at
org.apache.kafka.connect.util.TopicAdmin.verifyTopicCleanupPolicyOnlyCompact(TopicAdmin.java:420)
at
org.apache.kafka.connect.storage.KafkaConfigBackingStore$1.run(KafkaConfigBackingStore.java:501)
at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:133)
at
org.apache.kafka.connect.storage.KafkaConfigBackingStore.start(KafkaConfigBackingStore.java:268)
at
org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:130)
at
org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:288)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
I think the solution is to exclude mirror maker innner topic when auto creating
topics. (AutoTopicCreationManager.scala)
With this change, this problem already resolve for me.
was:
Hi Team
Mirror Maker: 2.7.0
when i enable auto create topic in both side:
auto.create.topics.enable=true
sometimes，mirror maker inner topic create by server not expected，mirror naker
start error.
```
[2021-11-19 18:03:56,707] ERROR [Worker clientId=connect-2, groupId=pr-mm2]
Uncaught exception in herder work thread, exiting:
(org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.common.config.ConfigException: Topic 'mm2-configs.pr.internal'
supplied via the 'config.storage.topic' property is required to have
'cleanup.policy=compact' to guarantee consistenc*****#*#*****connector
configurations, but found the topic currently has 'cleanup.policy=delete'.
Continuing would likely result in eventually losing connector configurations
and problems restarting this Connect cluster in the future. Change the
'config.storage.topic' property in the Connect worker configurations to use a
topic with 'cleanup.policy=compact'.
at
org.apache.kafka.connect.util.TopicAdmin.verifyTopicCleanupPolicyOnlyCompact(TopicAdmin.java:420)
at
org.apache.kafka.connect.storage.KafkaConfigBackingStore$1.run(KafkaConfigBackingStore.java:501)
at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:133)
at
org.apache.kafka.connect.storage.KafkaConfigBackingStore.start(KafkaConfigBackingStore.java:268)
at
org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:130)
at
org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:288)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
I think the solution is to exclude mirror maker innner topic when auto creating
topics.
(AutoTopicCreationManager.scala)
With this change, this problem already resolve for me.
> when auto create topics enable,server create inner topic of MirrorMaker
> unexpectedly
> -------------------------------------------------------------------------------------
>
> Key: KAFKA-13465
> URL: https://issues.apache.org/jira/browse/KAFKA-13465
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 2.7.0, 2.8.0, 2.7.1, 3.0.0
> Reporter: ZhenChun Pan
> Priority: Major
>
> Hi Team
> Mirror Maker: 2.7.0
> when i enable auto create topic in both side:
> auto.create.topics.enable=true
>
> sometimes，mirror maker inner topic create by server not expected，mirror naker
> start error.
> ```
> [2021-11-19 18:03:56,707] ERROR [Worker clientId=connect-2, groupId=pr-mm2]
> Uncaught exception in herder work thread, exiting:
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.common.config.ConfigException: Topic
> 'mm2-configs.pr.internal' supplied via the 'config.storage.topic' property is
> required to have 'cleanup.policy=compact' to guarantee
> consistenc****{*}#{*}#*****connector configurations, but found the topic
> currently has 'cleanup.policy=delete'. Continuing would likely result in
> eventually losing connector configurations and problems restarting this
> Connect cluster in the future. Change the 'config.storage.topic' property in
> the Connect worker configurations to use a topic with
> 'cleanup.policy=compact'.
> at
> org.apache.kafka.connect.util.TopicAdmin.verifyTopicCleanupPolicyOnlyCompact(TopicAdmin.java:420)
> at
> org.apache.kafka.connect.storage.KafkaConfigBackingStore$1.run(KafkaConfigBackingStore.java:501)
> at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:133)
> at
> org.apache.kafka.connect.storage.KafkaConfigBackingStore.start(KafkaConfigBackingStore.java:268)
> at
> org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:130)
> at
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:288)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ```
> I think the solution is to exclude mirror maker innner topic when auto
> creating topics. (AutoTopicCreationManager.scala)
> With this change, this problem already resolve for me.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
