[
https://issues.apache.org/jira/browse/KAFKA-13504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Katona updated KAFKA-13504:
----------------------------------
Description:
In case the Kafka Broker cluster and the Kafka Connect cluster is started
together and Connect would want to create its topics, there's a high chance to
fail the creation with InvalidReplicationFactorException.
{noformat}
ERROR org.apache.kafka.connect.runtime.distributed.DistributedHerder [Worker
clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder work
thread, exiting:
org.apache.kafka.connect.errors.ConnectException: Error while attempting to
create/find topic(s) 'connect-offsets'
...
Caused by: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication
factor: 3 larger than available brokers: 2.
{noformat}
Introducing a retry logic here would make Connect a bit more robust.
New configurations:
* offset.storage.topic.create.retries
* offset.storage.topic.create.retry.backoff.ms
* config.storage.topic.create.retries
* config.storage.topic.create.retry.backoff.ms
* status.storage.topic.create.retries
* status.storage.topic.create.retry.backoff.ms
was:In case the Kafka Broker cluster and the Kafka Connect cluster is started
together and Connect would want to create its topics, there's a high chance to
fail the creation with
> Retry connect internal topics' creation in case of
> InvalidReplicationFactorException
> ------------------------------------------------------------------------------------
>
> Key: KAFKA-13504
> URL: https://issues.apache.org/jira/browse/KAFKA-13504
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Andras Katona
> Assignee: Andras Katona
> Priority: Major
>
> In case the Kafka Broker cluster and the Kafka Connect cluster is started
> together and Connect would want to create its topics, there's a high chance
> to fail the creation with InvalidReplicationFactorException.
> {noformat}
> ERROR org.apache.kafka.connect.runtime.distributed.DistributedHerder [Worker
> clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder
> work thread, exiting:
> org.apache.kafka.connect.errors.ConnectException: Error while attempting to
> create/find topic(s) 'connect-offsets'
> ...
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication
> factor: 3 larger than available brokers: 2.
> {noformat}
> Introducing a retry logic here would make Connect a bit more robust.
> New configurations:
> * offset.storage.topic.create.retries
> * offset.storage.topic.create.retry.backoff.ms
> * config.storage.topic.create.retries
> * config.storage.topic.create.retry.backoff.ms
> * status.storage.topic.create.retries
> * status.storage.topic.create.retry.backoff.ms
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
