[
https://issues.apache.org/jira/browse/KAFKA-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695852#comment-17695852
]
Mickael Maison commented on KAFKA-14746:
----------------------------------------
https://issues.apache.org/jira/browse/KAFKA-14732 describes pretty much the
same issue. With the exponential backoff we don't spam the logs with errors.
However it's still kind of unclear what's going on from a user point of view
without looking at the logs.
If the connector throws in taskConfigs()
* If it's a reconfiguration, the existing tasks keep running (connector and
tasks show no errors in the REST API)
* If it's the creation of a new connector, the connector is marked as running
and no tasks are created.
In both cases, the runtime retries reconfiguring it forever.
> Throwing in Connector.taskConfigs generates a lot of logs
> ---------------------------------------------------------
>
> Key: KAFKA-14746
> URL: https://issues.apache.org/jira/browse/KAFKA-14746
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Mickael Maison
> Priority: Major
>
> If a Connector throws in its taskConfigs() method, the runtime ends up
> retrying using DistributedHerder.RECONFIGURE_CONNECTOR_TASKS_BACKOFF_MS which
> is a fixed value (250ms). For each retry, the runtime prints the connector
> configuration and the enriched configuration so this can quickly generate a
> lot of logs.
> There is some value in throwing in taskConfigs() as it allows to fail fast in
> case the connector is given bad credentials. For example this is what some of
> the Debezium connectors do:
> https://github.com/debezium/debezium/blob/main/debezium-connector-sqlserver/src/main/java/io/debezium/connector/sqlserver/SqlServerConnector.java#L56-L69
> The way Connectors are expected to work today is to instead always create
> tasks and let each task fail in case the configuration is wrong. We should
> document that and make it clear in the javadoc that throwing in taskConfigs
> is not recommended.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
