Any body ?? Any answers !!! I'm sure someone would have a work around !
Please help :) On Feb 15, 2014 12:11 AM, "Andrey Yegorov" <[email protected]> wrote: > I have exactly the same question. > I am using kafka spout from > https://github.com/wurstmeister/storm-kafka-0.8-plus.git with kafka 0.8 > release and ordinary (non-trident) storm topology. > > How can I guarantee processing of messages sent while topology was down or > while e.g. storm cluster was down for maintenance? > > ---------- > Andrey Yegorov > > > On Wed, Feb 12, 2014 at 8:05 AM, Danijel Schiavuzzi <[email protected]>wrote: > >> Hi Chitra, >> >> Which Kafka spout version are you exactly using, and what spout type -- >> Trident or the ordinary Storm spout? >> >> I ask that because, unfortunately, there are multiple Kafka spout >> versions around the web. According to my research, your best bet is the one >> in storm-contrib in case you use Kafka version 0.7, and >> storm-kafka-0.8-plus in case you use Kafka 0.8. >> >> Best regards, >> >> Danijel Schiavuzzi >> www.schiavuzzi.com >> >> >> On Wed, Feb 12, 2014 at 8:42 AM, Chitra Raveendran < >> [email protected]> wrote: >> >>> Hi >>> >>> I have a topology in production which uses the default kafka spout, I >>> have set this parameter >>> *spoutConfig.forceStartOffsetTime(-1);* >>> >>> This parameter -1 helps me in such a way that, it consumes from the >>> latest message, and doesn't start reading data from kafka right from the >>> beginning (That would be unnecessary and redundant in my usecase). >>> >>> But in production, whenever a new release goes in, I stop and start the >>> topology which would take a few seconds to minutes. I have been loosing out >>> on some data during the time that the topology is down. >>> >>> How can I avoid this. I have tried running without the ForcedOffsetTime >>> parameter, but that did not work. What am I doing wrong, how can I continue >>> reading from the last offest ? >>> >>> Thanks, >>> Chitra >>> >>> >> >> >> -- >> Danijel Schiavuzzi >> > >
