zhaobo created KAFKA-8543: ----------------------------- Summary: Optimize Fetch thread allocation strategy to ensure full utilization of Fetch thread resources Key: KAFKA-8543 URL: https://issues.apache.org/jira/browse/KAFKA-8543 Project: Kafka Issue Type: Improvement Components: core Affects Versions: 2.2.0 Reporter: zhaobo
The Follow TP on the Broker needs to fetch data from the Leader broker. The original fetch thread allocation strategy has two areas that can be optimized: (1)The configuration file has a parameter "num.consumer.fetchers" which limits the number of threads fetching data from a source broker. In fact, Fetch resources are not fully used, and only a part of the threads will be used with a high probability. When the traffic of the Leader TP increases, the fetch capability may be bottlenecked. (2)The number of TPs responsible for each Fetch thread may be uneven. Improve: We propose a polling allocation method to ensure that Fetch thread resources can be fully utilized. And the number of TPs responsible for each fetch thread is balanced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)