Hi Kafka devs, I'm researching the design decisions behind the Kafka consumer fetch logic in earlier versions (e.g., 2.4.0 and prior). Specifically, I'm interested in the rationale for the behavior where the consumer avoids sending a new fetch request if the internal buffer still contains unprocessed records. >From my understanding, this design prioritizes simplicity, predictable memory >usage, and avoidance of concurrency issues or corner cases-at the cost of >potentially lower maximum throughput. This behavior is described in JIRA >KAFKA-17182, but I'm looking for any explicit documentation, mailing list >discussions, or blog posts from Kafka architects or contributors that explain >this trade-off in detail. Could anyone point me to an official source or provide historical context on why this design was chosen?
Thanks, Felix Kocherovsky
