On that note - another error I see right around the same time is below - i see it on the consumer end. Note that in this case I have one consumer with a single iterator - and on each broker 8 partitions. When printing messages, I see an empty message on occasion...
Exception in thread "main" kafka.common.InvalidMessageSizeException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:355) at java.lang.Class.newInstance(Class.java:308) at kafka.common.ErrorMapping$.maybeThrowException(ErrorMapping.scala:53) at kafka.message.ByteBufferMessageSet.iterator(ByteBufferMessageSet.scala:60) at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:67) at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:30) at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:58) at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:50) at com.airbnb.logging.util.ConsumerTest.main(ConsumerTest.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
