[
https://issues.apache.org/jira/browse/KAFKA-14011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556337#comment-17556337
]
Jordan Moore edited comment on KAFKA-14011 at 6/20/22 11:32 AM:
----------------------------------------------------------------
"topic.creation" configs are only available for source connectors, for topics
that source connectors will write to, not dead letter topics. "admin" prefix
will only attempt to modify the AdminClient config, which does not include any
topic configs, such as retention time.
If you are allowing the broker to auto-create topics with defaults, then it's
recommended that you disable this, and create all necessary topics ahead of
time, with the topic settings you need.
For already existing topics, you can alter their configs.
Both operations can be done with kafka-topics script
was (Author: cricket007):
"topic.creation" configs are only available for source connectors, for topics
that source connectors will write to, not dead letter topics.
If you are allowing the broker to auto-create topics with defaults, then it's
recommended that you disable this, and create all necessary topics ahead of
time, with the topic settings you need.
For already existing topics, you can alter their configs.
Both operations can be done with kafka-topics script
> Reduce retention.ms from default 7 days to 1 day (Make it configurable) for
> the dead letter topic created for error handling via sink connector
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-14011
> URL: https://issues.apache.org/jira/browse/KAFKA-14011
> Project: Kafka
> Issue Type: Improvement
> Components: config, KafkaConnect
> Affects Versions: 3.0.1
> Reporter: Souvik
> Priority: Major
>
> We are creating a sink connector along with the error handling mechanism , so
> if in case if there is bad record it is routed to error queue , The
> properties used while creating the sink connector are as following
>
> 'errors.tolerance'='all',
> 'errors.deadletterqueue.topic.name' = 'error_dlq',
> 'errors.deadletterqueue.topic.replication.factor'= -1,
> 'errors.log.include.messages' = true,
> 'errors.log.enable' = true,
> 'errors.deadletterqueue.context.headers.enable' = true
> *now is there any way we can configure the retention.ms for the dead letter
> queue topic i.e from default 7 days to 1 day , or is there any way we can
> change the compaction policy by providing any property while creating the
> source connector*
> *we have tried the following properties in the connector*
> {code:java}
> 'topic.creation.default.retention.ms'='3000',
> 'admin.topic.creation.default.retention.ms'='3000',
> 'admin.retention.ms' = '3000',
> 'admin.topic.retention.ms' = '3000',
> 'admin.topic.creation.retention.ms' = '3000',
> 'error.topic.creation.default.retention.ms'='3000',
> 'error.deadletterqueue.topic.creation.default.retention.ms'='3000',
> 'error.deadletterqueue.topic.retention.ms'='3000',
> 'admin.topic.creation.default.retention.ms' = 3000, {code}
> but it did not work.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
