Souvik created KAFKA-14011: ------------------------------ Summary: Reduce retention.ms from default 7 days to 1 day (Make it configurable) for the dead letter topic created for error handling via sink connector Key: KAFKA-14011 URL: https://issues.apache.org/jira/browse/KAFKA-14011 Project: Kafka Issue Type: Improvement Components: config, KafkaConnect Affects Versions: 3.0.1 Reporter: Souvik
We are creating a sink connector along with the error handling mechanism , so if in case if there is bad record it is routed to error queue , The properties used while creating the sink connector are as following 'errors.tolerance'='all', 'errors.deadletterqueue.topic.name' = 'error_dlq', 'errors.deadletterqueue.topic.replication.factor'= -1, 'errors.log.include.messages' = true, 'errors.log.enable' = true, 'errors.deadletterqueue.context.headers.enable' = true *now is there any way we can configure the retention.ms for the dead letter queue topic i.e from default 7 days to 1 day , or is there any way we can change the compaction policy by providing any property while creating the source connector* *we have tried the following properties in the connector* {code:java} 'topic.creation.default.retention.ms'='3000', 'admin.topic.creation.default.retention.ms'='3000', 'admin.retention.ms' = '3000', 'admin.topic.retention.ms' = '3000', 'admin.topic.creation.retention.ms' = '3000', 'error.topic.creation.default.retention.ms'='3000', 'error.deadletterqueue.topic.creation.default.retention.ms'='3000', 'error.deadletterqueue.topic.retention.ms'='3000', 'admin.topic.creation.default.retention.ms' = 3000, {code} but it did not work. -- This message was sent by Atlassian Jira (v8.20.7#820007)