The only thing I notice in the logs which is a bit unsettling is about a once a second rate of messages of the type
"Closing socket connection to some-ip-address". I used to see these messages before but it seems like its more often than usual. Also all the clients that it seems to close connections with are running the Java wrapper over the Scala SimpleConsumer. Is there any logging I can enable to understand why exactly these connections are being closed so often? Thanks, Rajiv On Fri, Aug 21, 2015 at 3:50 PM, Rajiv Kurian <ra...@signalfuse.com> wrote: > We upgraded a 9 broker cluster from version 0.8.1 to version 0.8.2.1. > Actually we cherry-picked the commit > at 41ba26273b497e4cbcc947c742ff6831b7320152 to get zkClient 0.5 because we > ran into a bug described at > https://issues.apache.org/jira/browse/KAFKA-824 > > Right after the update the CPU spiked quite a bit but I am guessing that > is because the brokers were pulling in log segments from other brokers > right after restart. The CPU remained elevated for a while and I thought it > would come down after things settled down but the CPU has remained higher > even after a day. > > Our steady state CPU on the brokers went from about 28% (0.8.1) to 34% > (0.8.2.1). We do not use compression on any topic or partition. Our > incoming traffic (number of messages/sec) has not increased at all. Our > incoming bytes/sec has actually decreased because we managed to reduce the > size of one our message types from 256 bytes to 32 bytes. The message size > change was made hours after the Kafka version update and didn't seem to > harm or help the cpu. The bytes-in/sec and bytes-out/sec metrics have > definitely gone down after the message size reduction. > > Here is a link to the graph showing how the CPU went up - > http://i.imgur.com/KVJLzsX.png?1 The restarts were done from 18:00 to > 19:00 and I'd expect the CPU to go up at that time but I can't explain the > steady state CPU rise. > > Are there any known performance regressions after 0.8.1? Any hints on what > I should investigate if you think that this is not normal? > > Thanks, > Rajiv >
