adixitconfluent commented on code in PR #20746:
URL: https://github.com/apache/kafka/pull/20746#discussion_r2464537332
##########
server/src/main/java/org/apache/kafka/server/share/fetch/PartitionMaxBytesStrategy.java:
##########
@@ -58,7 +64,27 @@ static PartitionMaxBytesStrategy type(StrategyType type) {
private static LinkedHashMap<TopicIdPartition, Integer>
uniformPartitionMaxBytes(int requestMaxBytes, Set<TopicIdPartition> partitions,
int acquiredPartitionsSize) {
checkValidArguments(requestMaxBytes, partitions,
acquiredPartitionsSize);
LinkedHashMap<TopicIdPartition, Integer> partitionMaxBytes = new
LinkedHashMap<>();
- partitions.forEach(partition -> partitionMaxBytes.put(partition,
requestMaxBytes / acquiredPartitionsSize));
+ if (requestMaxBytes >= acquiredPartitionsSize) {
+ // Case 1: requestMaxBytes can be evenly distributed within
partitions.
+ partitions.forEach(partition -> partitionMaxBytes.put(partition,
requestMaxBytes / acquiredPartitionsSize));
+ } else if (requestMaxBytes >= partitions.size()) {
+ // Case 2: we will be distributing requestMaxBytes greedily in
this scenario to prevent any starvation.
+ partitions.forEach(partition -> partitionMaxBytes.put(partition,
requestMaxBytes / partitions.size()));
Review Comment:
>we introduced acquiredPartitionsSize to this method as
acquiredPartitionsSize can be greater than partitions size, in this method,
because there could be multiple independent reads occurring.
yes, this was introduced originally in order to prevent starvation of the
left topic partitions which will be fetched later as mentioned in the comment
https://github.com/apache/kafka/blob/trunk/core/src/main/java/kafka/server/share/DelayedShareFetch.java#L445
. However, if we continue to follow that logic, then the current set of
partitions are certainly going to starve. Hence, I greedily choose not to
starve these partitions. The approach taken in both case 1 and case 2 prevent
starvation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
