Hey everyone, I have a question in regards to the message consumption rate on Kafka 4.1.1 Recently I've updated a Kafka cluster from version 3.9.1 to 4.1.1 (just the cluster was updated - the client libraries are still on version 3.9.1). This cluster was already using KRaft, so the update was very much straight forward by just updating the version. After the upgrade though I've observed that the message rate by a (previously already very busy) Kafka consumer nearly halved.
I have mitigated the problem for now by putting more parallel consumption on it, but I'm wondering whether this is an expected thing or whether there was any change to Kafka that I'm unaware of. What should be known here is that each of my consumed messages is doing an in-sync manual offset commit. I know that this is not an ideal scenario to have a large throughput (e.g. using async commits or batched commits even). All I want to raise here is that the situation got worse compared to the version 3.9 and I'm wondering why that is. Additionally to note here is, that the offset replication factor is set to 3 and the min in-sync replicas is set to 2. Not sure if this is somehow connected and previously the offset commit already returned when one replica returned an ack and this now has changed (just guessing here). In any case I'm happy to hear your thoughts on this and thanks so much in advance. Kind regards Georg Friedrich
