This is just a conjecture though, one possible reason I can imagine is traffic increase => request queue full => request latency increase => more batching on client-side => overall request-count decrease => CPU-usage decrease. You should check request-count metric.
At any rate, full-request-queue situation is not desired so you should figure out the cause and address that for stable cluster operation. 2024年1月22日(月) 11:53 dong yu <hideonbus...@gmail.com>: > I have a question: why does the overall CPU of the cluster decrease when > the KAFKA cluster traffic increases, the request queue is full, and the > idle rate is low? > > Haruki Okada <ocadar...@gmail.com> 于2024年1月15日周一 21:56写道： > > > You should investigate the cause of request-queue full situation first. > > Since I guess low network idle ratio is the consequence of that. > > (Network-threads would block on queueing when request-queue is full) > > > > I recommend running async-profiler to take the profile of the broker > > process if possible (with wallclock mode). > > Then it will help you to identify the bottleneck which consuming > > request-handler time. > > > > 2024年1月15日(月) 17:58 dong yu <hideonbus...@gmail.com>: > > > > > This is my problem > > > 1.The request queue is always at 500 > > > 2.There are 130 machines in the cluster, and the network idle rate of > 30 > > > machines is less than 20. > > > > > > > > > This is my BROKER configuration > > > num.io.threads=32 > > > num.network.threads=64 > > > > > > How should I locate the problem? I tried to increase the parameters but > > the > > > effect was not obvious. > > > THKS。 > > > > > > > > > -- > > ======================== > > Okada Haruki > > ocadar...@gmail.com > > ======================== > > > -- ======================== Okada Haruki ocadar...@gmail.com ========================
