[
https://issues.apache.org/jira/browse/KAFKA-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282875#comment-17282875
]
Goltseva Taisiia commented on KAFKA-10340:
------------------------------------------
And also I cannot understand, why we even have the property
"topic.creation.enable" and do not create all topics by AdminClient? Isn't it a
better approach to create topics always explicitly using AdminClient than to
rely on auto creation?
And also I cannot see how can it has a bad impact on users. Why someone may
want to set "topic.creation.enable = false"?
According to KIP-158 we have an option to set replication.factor and partitions
to -1 to use the broker's default value. So why we cannot always create topics
by AdminClient and use these broker's default values in case connector does not
have any "topic.creation" properties?
As far as I understand this will completely solve the problem described in this
ticket. And also people will be able to set "auto.create.topics.enable = false"
on Kafka cluster without any impact.
Please, correct me if I am wrong.
> Source connectors should report error when trying to produce records to
> non-existent topics instead of hanging forever
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-10340
> URL: https://issues.apache.org/jira/browse/KAFKA-10340
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: Arjun Satish
> Assignee: Chris Egerton
> Priority: Major
>
> Currently, a source connector will blindly attempt to write a record to a
> Kafka topic. When the topic does not exist, its creation is controlled by the
> {{auto.create.topics.enable}} config on the brokers. When auto.create is
> disabled, the producer.send() call on the Connect worker will hang
> indefinitely (due to the "infinite retries" configuration for said producer).
> In setups where this config is usually disabled, the source connector simply
> appears to hang and not produce any output.
> It is desirable to either log an info or an error message (or inform the user
> somehow) that the connector is simply stuck waiting for the destination topic
> to be created. When the worker has permissions to inspect the broker
> settings, it can use the {{listTopics}} and {{describeConfigs}} API in
> AdminClient to check if the topic exists, the broker can
> {{auto.create.topics.enable}} topics, and if these cases do not exist, either
> throw an error.
> With the recently merged
> [KIP-158|https://cwiki.apache.org/confluence/display/KAFKA/KIP-158%3A+Kafka+Connect+should+allow+source+connectors+to+set+topic-specific+settings+for+new+topics],
> this becomes even more specific a corner case: when topic creation settings
> are enabled, the worker should handle the corner case where topic creation is
> disabled, {{auto.create.topics.enable}} is set to false and topic does not
> exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
