forceStartOffsetTime value can be -2, -1, or a time stamp in milliseconds
- -1 to read the latest offset of the topic - -2 to read from the beginning. - timestamp to read from a specific time On Sun, Feb 16, 2014 at 6:15 PM, Chitra Raveendran < [email protected]> wrote: > Any body ?? Any answers !!! > > I'm sure someone would have a work around ! > > Please help :) > On Feb 15, 2014 12:11 AM, "Andrey Yegorov" <[email protected]> > wrote: > >> I have exactly the same question. >> I am using kafka spout from >> https://github.com/wurstmeister/storm-kafka-0.8-plus.git with kafka 0.8 >> release and ordinary (non-trident) storm topology. >> >> How can I guarantee processing of messages sent while topology was down >> or while e.g. storm cluster was down for maintenance? >> >> ---------- >> Andrey Yegorov >> >> >> On Wed, Feb 12, 2014 at 8:05 AM, Danijel Schiavuzzi >> <[email protected]>wrote: >> >>> Hi Chitra, >>> >>> Which Kafka spout version are you exactly using, and what spout type -- >>> Trident or the ordinary Storm spout? >>> >>> I ask that because, unfortunately, there are multiple Kafka spout >>> versions around the web. According to my research, your best bet is the one >>> in storm-contrib in case you use Kafka version 0.7, and >>> storm-kafka-0.8-plus in case you use Kafka 0.8. >>> >>> Best regards, >>> >>> Danijel Schiavuzzi >>> www.schiavuzzi.com >>> >>> >>> On Wed, Feb 12, 2014 at 8:42 AM, Chitra Raveendran < >>> [email protected]> wrote: >>> >>>> Hi >>>> >>>> I have a topology in production which uses the default kafka spout, I >>>> have set this parameter >>>> *spoutConfig.forceStartOffsetTime(-1);* >>>> >>>> This parameter -1 helps me in such a way that, it consumes from the >>>> latest message, and doesn't start reading data from kafka right from the >>>> beginning (That would be unnecessary and redundant in my usecase). >>>> >>>> But in production, whenever a new release goes in, I stop and start the >>>> topology which would take a few seconds to minutes. I have been loosing out >>>> on some data during the time that the topology is down. >>>> >>>> How can I avoid this. I have tried running without the ForcedOffsetTime >>>> parameter, but that did not work. What am I doing wrong, how can I continue >>>> reading from the last offest ? >>>> >>>> Thanks, >>>> Chitra >>>> >>>> >>> >>> >>> -- >>> Danijel Schiavuzzi >>> >> >> -- With Regards, Vinoth Kumar K
