Hi! I have written a guide how to calculate the minimal speed of Kafka consumers - so you can avoid the situation where your Kafka consumers are to slow, and because of this you lose data. The guide takes into account three key factors - producer speed, retention period of your data in Kafka and the amount of downtime you plan to have.
Can you please check it out and and see if it makes sense. Feedback appreciated :) Link: https://dpoldrugo.github.io/2016/12/29/dimensioning-kafka-consumers Davor Poldrugo https://dpoldrugo.github.io
