[jira] [Commented] (KAFKA-9039) Optimize replica fetching CPU utilization with large number of partitions

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/KAFKA-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952996#comment-16952996
 ] 

ASF GitHub Bot commented on KAFKA-9039:
---

guozhangwang commented on pull request #7443: KAFKA-9039: Optimize 
ReplicaFetcher fetch path
URL: https://github.com/apache/kafka/pull/7443
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Optimize replica fetching CPU utilization with large number of partitions
> -
>
> Key: KAFKA-9039
> URL: https://issues.apache.org/jira/browse/KAFKA-9039
> Project: Kafka
>  Issue Type: Improvement
>  Components: core
>Reporter: Guozhang Wang
>Priority: Major
>
> Our current follower replica fetching logic has huge CPU cost with 
> num.partitions to fetch from, and it scales non-linearly as well. There are a 
> bunch of optimizations we can consider to try to reduce its cost and 
> hopefully make it to be linear against the num.partitions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (KAFKA-9039) Optimize replica fetching CPU utilization with large number of partitions

[Guozhang Wang (Jira)]

[ 
https://issues.apache.org/jira/browse/KAFKA-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951186#comment-16951186
 ] 

Guozhang Wang commented on KAFKA-9039:
--

Here's a first PR against this ticket: https://github.com/apache/kafka/pull/7443

> Optimize replica fetching CPU utilization with large number of partitions
> -
>
> Key: KAFKA-9039
> URL: https://issues.apache.org/jira/browse/KAFKA-9039
> Project: Kafka
>  Issue Type: Improvement
>  Components: core
>Reporter: Guozhang Wang
>Priority: Major
>
> Our current follower replica fetching logic has huge CPU cost with 
> num.partitions to fetch from, and it scales non-linearly as well. There are a 
> bunch of optimizations we can consider to try to reduce its cost and 
> hopefully make it to be linear against the num.partitions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)