Re: Kafka cluster performance degradation (Kafka 0.8.2.1)

Hi Cliff, Thanks for getting back to me on this. Sorry for the delay on replying to you, but I was away on holiday. I have been monitoring the Kafka brokers via JMX and on all nodes CPU usage was pretty much idle, under 1% as you said. I should have mentioned before that the Kafka cluster was

Re: Kafka cluster performance degradation (Kafka 0.8.2.1)

Hi Leo, We also have a fairly idle CPU for our brokers, but we aren't tracking latency like you are. We are mostly interested in overall throughput and storage. We are using compression and bundling which add to latency but decrease storage usage. Good luck, Cliff On Wed, Feb 3, 2016 at 10:55

Kafka cluster performance degradation (Kafka 0.8.2.1)

Hi all, We are using Kafka in production and we have been facing some performance degradation of the cluster, apparently after the cluster is a bit "old". We have our production cluster which is up and running since 31/12/2015 and performance tests on our application measuring a full round

Re: Kafka cluster performance degradation (Kafka 0.8.2.1)

Hi Leo, I'm not sure if this is the issue you're encountering, but this is what we found when we went from 0.8.1.1 to 0.8.2.1. Snappy compression didn't work as expected. Something in the library broke compressing bundles of messages and each message was compressed individually (which for us

Re: Kafka cluster performance degradation (Kafka 0.8.2.1)

We fixed two serious Snappy bugs for 0.8.2.2, so you may want to check that. On Thu, Jan 21, 2016 at 8:16 AM, Cliff Rhyne wrote: > Hi Leo, > > I'm not sure if this is the issue you're encountering, but this is what we > found when we went from 0.8.1.1 to 0.8.2.1. > > Snappy