[
https://issues.apache.org/jira/browse/KAFKA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-13549:
------------------------------------
Labels: kip (was: needs-kip)
> Add "delete interval" config
> ----------------------------
>
> Key: KAFKA-13549
> URL: https://issues.apache.org/jira/browse/KAFKA-13549
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Priority: Major
> Labels: kip
>
> Kafka Streams uses "delete record" requests to aggressively purge data from
> repartition topics. Those request are sent each time we commit.
> For at-least-once with a default commit interval of 30 seconds, this works
> fine. However, for exactly-once with a default commit interval of 100ms, it's
> very aggressive. The main issue is broker side, because the broker logs every
> "delete record" request, and thus broker logs are spammed if EOS is enabled.
> We should consider to add a new config (eg `delete.record.interval.ms` or
> similar) to have a dedicated config for "delete record" requests, to decouple
> it from the commit interval config and allow to purge data less aggressively,
> even if the commit interval is small to avoid the broker side log spamming.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
