I see (and I think I understand why it works this way). I see that kafka-87 addresses this with a request for having a time based index, this would be relatively useful, but I also would like to have a way to go back say 1,000 messages. Other than walking backwards one segment at a time, can then scanning forward from there, do you have any suggestions how this might be done or is it also a feature request?
On Wed, Sep 21, 2011 at 5:14 PM, Joel Koshy <jjko...@gmail.com> wrote: > Hi Taylor, > > This is an FAQ, that was asked some time ago as well: > > http://mail-archives.apache.org/mod_mbox/incubator-kafka-users/201108.mbox/%3ccafbh0q3qgsyxn4kjtth0zz9djz7j14tm5g24rfjsihffbqd...@mail.gmail.com%3E > > GetOffsetShell doesn't return the offset of every message. It returns the > offset of the first message in every segment file. If you provide a time, > the offsets you get back are based on the last modified time of the segment > files. > > It may be a good idea to paraphrase the above to be displayed with the > tool's help message. > > Thanks, > > Joel > > On Wed, Sep 21, 2011 at 4:18 PM, Taylor Gautier <tgaut...@tagged.com> > wrote: > > Hi, > > > > Using Kafka 0.6. > > > > I'm trying to use get offsets but it doesn't seem to work as I expect. I > > have a test topic that has some messages in it. Here's the output of a > > test client that starts from offset 0 and prints all messages/offsets for > > the topic: > > > > Consumed message:foo offset: 12 > > Consumed message:bar offset: 24 > > Consumed message:foo offset: 36 > > Consumed message:bar offset: 48 > > Consumed message:hello offset: 62 > > Consumed message:world offset: 76 > > > > Here's a class to print the last n offsets: > > > > public class SimpleConsumerDemo > > > > { > > > > public static void main(String[] args) > > > > { > > > > SimpleConsumer simpleConsumer = new SimpleConsumer("localhost", 9092, > > 1000, 1024); > > > > > > long[] offsets = simpleConsumer.getOffsetsBefore("test", 0, -1L, 1); > > > > for (long l : offsets) { > > > > System.out.println("offset: " + l); > > > > } > > > > } > > > > } > > > > > > Running this as above, with 1 offset, yields expected results: > > > > ----output----- > > > > offset: 76 > > > > > > However, asking for 3 offsets yields unexpected results: > > > > > > change to: long[] offsets = simpleConsumer.getOffsetsBefore("test", 0, > -1L, > > 3); > > > > ----output----- > > > > offset: 76 > > > > offset: 0 > > > > > > I expected: > > > > > > ----output----- > > > > offset: 76 > > > > offset: 62 > > > > offset: 48 > > > > Any idea why I did not get what I was looking for/what I am doing wrong? > > >
