kafka relies on the underlying OS' page cache for serving "popular" data. so "pre-assembling" push batches would move from page cache to heap storage, which is not as appealing. also, for trivial cases a lot of consumers read the same thing, which would make the heap caching even worse.
also - i dont think you need to shorten fetch.max.wait.ms to get lower delays - you could still configure a relatively long fetch.max.wait.ms and have the broker answer your poll the minute _any_ messags are available. On Wed, Feb 1, 2017 at 2:46 AM, Alexander Binzberger < alexander.binzber...@wingcon.com> wrote: > ave very few. I don't see how push would cost more CPU time or resources > on the broker then polling with a lot of consumers very frequently.
