We're using 0.8.2.1 processing maybe 1 million messages per hour. Each message includes tracking information with a timestamp for when it was produced, and a timestamp for when it was consumed, to give us roughly the amount of time it spent in Kafka. On average this number is in the seconds and our upper percentiles are in the minutes.
What metrics and settings can we look at to figure out why we might be spending so much time in Kafka?
