Yes I tried without the forced offset time parameter, but the topology stopped consuming messages.
Regards Chitra On Feb 17, 2014 1:24 AM, "P. Taylor Goetz" <[email protected]> wrote: > If turn off forceOffsetTime, it should resume from the last offset stored > in zookeeper. > > - Taylor > > On Feb 16, 2014, at 12:35 PM, Chitra Raveendran < > [email protected]> wrote: > > Hi > > So according to this logic I should set the timestamp parameter to the > value when the topology was stopped ? > > But how do we identify the exact instance when the topology went down, so > that storm could start consuming from then ? Is it based on approximation ? > Or is there some concrete way to find the exact instance when the topology > was down? > > Is there any other parameter which is based on last offset and not time? > > Regards > Chitra > On Feb 16, 2014 11:00 PM, "Vinoth Kumar Kannan" <[email protected]> > wrote: > >> forceStartOffsetTime value can be -2, -1, or a time stamp in >> milliseconds >> >> >> - -1 to read the latest offset of the topic >> - -2 to read from the beginning. >> - timestamp to read from a specific time >> >> >> >> On Sun, Feb 16, 2014 at 6:15 PM, Chitra Raveendran < >> [email protected]> wrote: >> >>> Any body ?? Any answers !!! >>> >>> I'm sure someone would have a work around ! >>> >>> Please help :) >>> On Feb 15, 2014 12:11 AM, "Andrey Yegorov" <[email protected]> >>> wrote: >>> >>>> I have exactly the same question. >>>> I am using kafka spout from >>>> https://github.com/wurstmeister/storm-kafka-0.8-plus.git with kafka >>>> 0.8 release and ordinary (non-trident) storm topology. >>>> >>>> How can I guarantee processing of messages sent while topology was down >>>> or while e.g. storm cluster was down for maintenance? >>>> >>>> ---------- >>>> Andrey Yegorov >>>> >>>> >>>> On Wed, Feb 12, 2014 at 8:05 AM, Danijel Schiavuzzi <[email protected] >>>> > wrote: >>>> >>>>> Hi Chitra, >>>>> >>>>> Which Kafka spout version are you exactly using, and what spout type >>>>> -- Trident or the ordinary Storm spout? >>>>> >>>>> I ask that because, unfortunately, there are multiple Kafka spout >>>>> versions around the web. According to my research, your best bet is the >>>>> one >>>>> in storm-contrib in case you use Kafka version 0.7, and >>>>> storm-kafka-0.8-plus in case you use Kafka 0.8. >>>>> >>>>> Best regards, >>>>> >>>>> Danijel Schiavuzzi >>>>> www.schiavuzzi.com >>>>> >>>>> >>>>> On Wed, Feb 12, 2014 at 8:42 AM, Chitra Raveendran < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> I have a topology in production which uses the default kafka spout, I >>>>>> have set this parameter >>>>>> *spoutConfig.forceStartOffsetTime(-1);* >>>>>> >>>>>> This parameter -1 helps me in such a way that, it consumes from the >>>>>> latest message, and doesn't start reading data from kafka right from the >>>>>> beginning (That would be unnecessary and redundant in my usecase). >>>>>> >>>>>> But in production, whenever a new release goes in, I stop and start >>>>>> the topology which would take a few seconds to minutes. I have been >>>>>> loosing >>>>>> out on some data during the time that the topology is down. >>>>>> >>>>>> How can I avoid this. I have tried running without the >>>>>> ForcedOffsetTime parameter, but that did not work. What am I doing wrong, >>>>>> how can I continue reading from the last offest ? >>>>>> >>>>>> Thanks, >>>>>> Chitra >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Danijel Schiavuzzi >>>>> >>>> >>>> >> >> >> -- >> With Regards, >> Vinoth Kumar K >> > >
