Re: log.cleaner.io.max.bytes.per.second

2022-12-28 Thread Luke Chen
Hi Nicolas, > Any Idea some may have to limit log compaction disk usage ? That's a pretty big issue we're having, compaction of a few topics is using far more IO than everything else on the broker... I don't know any other better solution except make the `log.cleaner.io.max.bytes.per.second` lowe

Re: log.cleaner.io.max.bytes.per.second

2022-12-28 Thread Nicolas Carlot
Okay, thx for the explaination Luc. When looking for logs on a bigger topic, I can see the IO limitation kinda works. I have like15MB/s when having configured log.cleaner.io.max.bytes.per.second=1024.0 I'm not very comfortable with this. Setting the limit rate to 1Kb/s, ending with 15Mb/s, seems a

Re: log.cleaner.io.max.bytes.per.second

2022-12-28 Thread Luke Chen
Hi Nicolas, The throttler in log cleaner is just a simple throttler to control the IO speed by periodically (every 300ms) check the IO rate so far, and sleep some time to slow it down. That is, in your case, it could be: log cleaner runs for 300ms with full IO speed -> check -> throttle (sleep to