Could you run the DumpLogSegments tool to see if there are indeed corrupted messages in the broker log?
Thanks, Jun On Mon, May 12, 2014 at 4:08 PM, Xuyen On <x...@ancestry.com> wrote: > Hi all, > > I am seeing some corrupt data on my 0.7.2 Kafka cluster. Every once in > awhile I'll get the following message: > > Exception in thread "kafka-consumer" java.lang.IndexOutOfBoundsException > at java.nio.Buffer.checkIndex(Unknown Source) > at java.nio.HeapByteBuffer.get(Unknown Source) > at kafka.message.Message.magic(Message.scala:133) > at kafka.message.Message.checksum(Message.scala:146) > at kafka.message.Message.isValid(Message.scala:158) > at > kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet.scala:129) > at > kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:166) > at > kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:100) > at > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59) > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51) > at scala.collection.Iterator$class.foreach(Iterator.scala:631) > at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30) > at > scala.collection.IterableLike$class.foreach(IterableLike.scala:79) > at kafka.message.MessageSet.foreach(MessageSet.scala:87) > at > kafka.tools.SimpleConsumerShell$$anon$1$$anonfun$run$1.apply(SimpleConsumerShell.scala:97) > at > kafka.tools.SimpleConsumerShell$$anon$1$$anonfun$run$1.apply(SimpleConsumerShell.scala:94) > at scala.collection.Iterator$class.foreach(Iterator.scala:631) > at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30) > at > scala.collection.IterableLike$class.foreach(IterableLike.scala:79) > at > kafka.api.MultiFetchResponse.foreach(MultiFetchResponse.scala:25) > at > kafka.tools.SimpleConsumerShell$$anon$1.run(SimpleConsumerShell.scala:94) > > The only way I can get past this error is to move the offset past the > current message to skip the bad data. I use the > simpleConsumer.getOffsetsBefore() method to get a valid lists of offsets > and I skip to the next valid offset. > > Has anyone encountered this issue before? I found a message where someone > experienced similar issues and they were running JDK 1.7 and was able to > get around it by running JDK 1.6. I am running OpenJDK 1.7 but I tried > running Sun JDK 1.6 but I still have the same problem. > > My guess is that the data is corrupt somehow but I don't know how to look > at the raw data to confirm. Can anyone suggest ideas for me to debug this > issue? > > Thanks, > > Xuyen > > >
