Mickael Maison created KAFKA-14980:
--------------------------------------
Summary: MirrorMaker consumers don't get configs prefixed with
source.cluster
Key: KAFKA-14980
URL: https://issues.apache.org/jira/browse/KAFKA-14980
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 3.5.0
Reporter: Mickael Maison
As part of KAFKA-14021, we made a change to
MirrorConnectorConfig.sourceConsumerConfig() to grab all configs that start
with "source.". Previously it was grabbing configs prefixed with
"source.cluster.".
This means existing connector configuration stop working, as configurations
such as bootstrap.servers are not passed to source consumers.
For example, the following connector configuration was valid in 3.4 and now
makes the connector tasks fail:
{code:json}
{
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"name": "source",
"topics": "test",
"tasks.max": "30",
"source.cluster.alias": "one",
"target.cluster.alias": "two",
"source.cluster.bootstrap.servers": "localhost:9092",
"target.cluster.bootstrap.servers": "localhost:29092"
}
{code}
The connector attempts to start source consumers with bootstrap.servers = []
and the task crash with
{noformat}
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:837)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:671)
at
org.apache.kafka.connect.mirror.MirrorUtils.newConsumer(MirrorUtils.java:59)
at
org.apache.kafka.connect.mirror.MirrorSourceTask.start(MirrorSourceTask.java:103)
at
org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.initializeAndStart(AbstractWorkerSourceTask.java:274)
at
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
at
org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:75)
at
org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:181)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable
bootstrap urls given in bootstrap.servers
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
