Xi Wang created KAFKA-20917:
-------------------------------
Summary: Improve RecordAccumulator.ready performance
Key: KAFKA-20917
URL: https://issues.apache.org/jira/browse/KAFKA-20917
Project: Kafka
Issue Type: Improvement
Components: producer
Reporter: Xi Wang
Attachments: Screenshot 2026-08-10 at 8.37.01 PM.png, Screenshot
2026-08-10 at 8.42.36 PM.png, Screenshot 2026-08-10 at 8.45.57 PM.png,
image-2026-08-10-20-35-09-732.png, image-2026-08-10-20-43-01-781.png
After upgrading producer client from 2.9 to 3.9, we noticed produce latency
increase, especially for ack-all producer, e.g. P50 latency increase from 8ms
to 13ms, ~60% increase.
!Screenshot 2026-08-10 at 8.37.01 PM.png!
The profiling shows partitionReady call is hot.
!image-2026-08-10-20-35-09-732.png!
The 3.9 client include many checks and calculations in RecordAccumulator.ready
-> partitionReady method, especially for adaptive partitioning. after disable
adaptive partitioning, we still didn't see latency drop to as 2.9 client.
2.9 client profiling
!Screenshot 2026-08-10 at 8.42.36 PM.png!
3.9 client profiling
!image-2026-08-10-20-43-01-781.png!
We found out the linked PRs address performance issue and after deploying it to
our ack-all cluster (with adaptive partitioning disabled), the latency became
similar as 2.9 client.
!Screenshot 2026-08-10 at 8.45.57 PM.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
