Hi Guys, We're getting this same issue. It seems that this thread petered out with the suggestion to try 0.7.
Is there anything we can try with 0.6 to fix this? Thanks, Matt -----Original Message----- From: Nalin DeZoysa [mailto:ndezo...@tagged.com] Sent: Thursday, November 03, 2011 12:53 PM To: kafka-users@incubator.apache.org Subject: Re: Topics getting corrupted Hi Neha, No this isn't a valid offset. Right now its not possible to test with 0.7 because of the new message format, and i'm using a node.js kafka client which doesn't support this yet. Do you know of a workaround that i can use until I can test with the new version to get it to work/recover ? I'm running with Kafka 0.6 Thanks, Nalin On Thu, Nov 3, 2011 at 12:28 PM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > Nalin, > > >> kafka.common.OffsetOutOfRangeException: offset 281474976724790 is > >> out > of range > > Is this offset a valid one ? If you check the logs for that topic, do > you see this offset ? > > If yes, then it could be a bug in the message set iterator. > > >> > also the command that worked for me was ./bin/kafka-run-class.sh > kafka.utils.DumpLogSegments log_file > > That means, you are running on an older version. Do you mind trying > this out with 0.7 RC5 ? > > Thanks, > Neha > > On Thu, Nov 3, 2011 at 11:58 AM, Nalin DeZoysa <ndezo...@tagged.com> > wrote: > > Ran the util, log looks fine. Seems like the issue isn't with the > > log > file > > itself. The issue does not resolve itself even though i restart > > kafka and zookeeper, or run kafka without zookeeper. > > > > also the command that worked for me was ./bin/kafka-run-class.sh > > kafka.utils.DumpLogSegments log_file > > > > -Nalin > > > > On Thu, Nov 3, 2011 at 11:40 AM, Neha Narkhede > ><neha.narkh...@gmail.com > >wrote: > > > >> Nalin, > >> > >> You can use it like this - > >> ./bin/kafka-run-class.sh kafka.tools.DumpLogSegments log_file > >> --noprint > >> > >> This will print the offset of each message, along with other attributes. > >> > >> We haven't really standardized this tool, in spite of being very useful. > >> > >> Thanks, > >> Neha > >> > >> On Thu, Nov 3, 2011 at 11:33 AM, Nalin DeZoysa > >> <ndezo...@tagged.com> > >> wrote: > >> > Could you tell me how to run DumpLogSegment? > >> > > >> > > >> > Thanks, > >> > Nalin > >> > > >> > On Wed, Nov 2, 2011 at 6:19 PM, Jun Rao <jun...@gmail.com> wrote: > >> > > >> >> I don't see the attached log file. I guess the Apache mailing > >> >> list > >> doesn't > >> >> really allow attachments. Could you run the util > >> >> DumpLogSegement? It > >> will > >> >> tell you whether the log file is corrupted or not. > >> >> > >> >> Thanks, > >> >> > >> >> Jun > >> >> > >> >> On Wed, Nov 2, 2011 at 3:54 PM, Nalin DeZoysa > >> >> <ndezo...@tagged.com> > >> wrote: > >> >> > >> >> > Hi, > >> >> > > >> >> > I'm seeing that certain topics get corrupted - I can publish > >> >> > to it, > >> but > >> >> > when I try and subscribe to a topic, the response that comes > >> >> > back > has > >> >> > header error code 1. I also see this exception being thrown by > Kafka. > >> >> > > >> >> > [2011-11-02 15:36:40,947] ERROR error when processing request > >> >> > topic:cards_card_251791, part:0 offset:281474976724790 > maxSize:1048576 > >> >> > (kafka.server.KafkaRequestHandlers) > >> >> > kafka.common.OffsetOutOfRangeException: offset 281474976724790 > >> >> > is > out > >> of > >> >> > range > >> >> > at kafka.log.Log$.findRange(Log.scala:47) > >> >> > at kafka.log.Log.read(Log.scala:223) at > >> >> > > >> >> > >> > kafka.server.KafkaRequestHandlers.kafka$server$KafkaRequestHandlers$$r > eadMessageSet(KafkaRequestHandlers.scala:124) > >> >> > at > >> >> > > >> >> > >> > kafka.server.KafkaRequestHandlers.handleFetchRequest(KafkaRequestHandl > ers.scala:107) > >> >> > at > >> >> > > >> >> > >> > kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$2.apply(KafkaReq > uestHandlers.scala:42) > >> >> > at > >> >> > > >> >> > >> > kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$2.apply(KafkaReq > uestHandlers.scala:42) > >> >> > at kafka.network.Processor.handle(SocketServer.scala:268) > >> >> > at kafka.network.Processor.read(SocketServer.scala:291) > >> >> > at kafka.network.Processor.run(SocketServer.scala:202) > >> >> > at java.lang.Thread.run(Thread.java:619) > >> >> > > >> >> > > >> >> > > >> >> > I have attached the log file for the topic. > >> >> > > >> >> > Is there a way to work around this issue? > >> >> > > >> >> > > >> >> > Thanks, > >> >> > Nalin > >> >> > > >> >> > >> > > >> > > >
