I think you can use kafkaConfig.forceFromStart = *false*; We have implemented this and its working fine.
Regards, Riyaz On Thu, May 29, 2014 at 1:02 AM, Raphael Hsieh <[email protected]> wrote: > This is still not working for me. I've set the offset to -1 and it is > still backfilling data. > Is there any documentation on the start offsets that I could take a look > at ? > Or even documentation on kafka.api.OffsetRequest.LatestTime() ? > > > On Wed, May 28, 2014 at 1:01 PM, Raphael Hsieh <[email protected]>wrote: > >> would the Trident version of this be >> tridentKafkaConfig.startOffsetTime ? >> >> >> On Wed, May 28, 2014 at 12:23 PM, Danijel Schiavuzzi < >> [email protected]> wrote: >> >>> By default, the Kafka spout resumes consuming where it last left off. >>> That offset is stored in ZooKeeper. >>> >>> You can set forceStartOffset to -2 to start consuming from the earliest >>> available offset, or -1 to start consuming from the latest available offset. >>> >>> >>> On Wednesday, May 28, 2014, Raphael Hsieh <[email protected]> wrote: >>> >>>> If I don't tell trident to start consuming data from the beginning of >>>> the Kafka stream, where does it start from? >>>> If I were to do: >>>> tridentKafkaConfig.forceFromStart = true; >>>> Then it will tell the spout to start consuming from the start of the >>>> stream. If that is not set, then where does it start consuming from? and >>>> How might I go about telling it to start consuming from the very end of the >>>> stream? >>>> >>>> If a disaster were to happen and all my hosts died, when I start my >>>> cluster back up, it might start consuming from where it left off. I would >>>> rather manually process that old data, and have my storm system start >>>> processing the live data. >>>> >>>> Thanks >>>> -- >>>> Raphael Hsieh >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Danijel Schiavuzzi >>> >>> E: [email protected] >>> W: www.schiavuzzi.com >>> T: +385989035562 >>> Skype: danijels7 >>> >> >> >> >> -- >> Raphael Hsieh >> Amazon.com >> Software Development Engineer I >> (978) 764-9014 >> >> >> >> > > > > -- > Raphael Hsieh > Amazon.com > Software Development Engineer I > (978) 764-9014 > > > > -- Regards, Riyaz
