Re: Disk space - sharp increase in usage

WMF recently had an issue where Kafka broker disks were filling up with log segment data. It turned out that Kafka was not deleting old log segments because the oldest log segment had a message with a Kafka timestamp a year in the future. Since

Re: Disk space - sharp increase in usage

Hi Victoria, There are no metrics of when a config was changed. However, if you've been capturing the JMX metrics from the brokers, the metric kafka.cluster:name=ReplicasCount,partition=*,topic=*,type=Partition would show if replication factor was increased. As for retention time, if you're sure

Re: Disk space - sharp increase in usage

Regards kafka-logs directory, it was an interesting lead, we checked and it is the same. Regards replication factor and retention, I am not looking for current information, I am look for metrics that can give me information about a change. Still looking for more ideas On 02/06/2020, 11:31,

Re: Disk space - sharp increase in usage

> On Jun 2, 2020, at 12:56 AM, Victoria Zuberman > wrote: > > Hi, > > Background: > Kafka cluster > 7 brokers, with 4T disk each > version 2.3 (recently upgraded from 0.1.0 via 1.0.1) > > Problem: > Used disk space went from 40% to 80%. > Looking for root cause. > > Suspects: > > 1.