That's unusually large. Ours are around 32k-90mb each. Initially curious if you have log.cleaner.enable=true and what offsets.retention.minutes is set to.
And yes it can affect cluster performance. We had instances of consumer outages that were caused by bugged large consumer offfset files, especially during broker restarts where that partition had an index repair, or during leadership changes, correlating with a large IO spike where presumably the whole partition is read. for a given large partition, are the replicas the same size in each broker? It's possible for the log cleaner thread to die which needs a broker restart to possibly fix, and investigation if it keeps dying. Do you see any errors in your logs around that topic-partition? You can check (and monitor) the JMX metric for kafka.log:type=LogCleanerManager,name=time-since-last-run-ms which if the cleaner thread has died should be quite high. Our is normally around 15s. ps: https://twitter.com/gwenshap/status/1026888706429353986 On Wed, Sep 19, 2018 at 6:25 AM Liam Clarke <liam.cla...@adscale.co.nz> wrote: > Odd that the log compaction isn't working. What OS is your broker running > on and can you please post your server.properties? > > On Wed, 19 Sep. 2018, 2:13 am Kaushik Nambiar, <kaushiknambia...@gmail.com > > > wrote: > > > > > > > Hello, > > > We have a Kafka 0.11.xx version setup. > > > So the system topic which is __consumer_offset, we are looking at many > > > such topics like __consumer_offset-1,2,4..... > > > So one topic in particular,is now having log segments which is > > > contributing to 5GB of data. > > > I had a look at our server.properties file but couldnot find a property > > > called offsets.retention.minutes (found via Google). > > > So we are assuming Kafka will take the default value for such > properties. > > > So based on the default value of offset.retention.minutes property > which > > > is I guess 24 hours,but still we can see log segments older than few > > month. > > > So my question is ,is it safe to have that consumer_offset-7 topic > > (having > > > 5GB of log segments) deleted.Would it affect kafka anyway that would b > a > > > problem in production environment. > > > If deleting the topics is not a safe implementation,can you suggest a > > > different fix for this? > > > > > > > > > Thanks and Regards, > > > Kaushik Nambiar > > > > > >
