This is still not working for me. I've set the offset to -1 and it is still backfilling data. Is there any documentation on the start offsets that I could take a look at ? Or even documentation on kafka.api.OffsetRequest.LatestTime() ?
On Wed, May 28, 2014 at 1:01 PM, Raphael Hsieh <[email protected]> wrote: > would the Trident version of this be > tridentKafkaConfig.startOffsetTime ? > > > On Wed, May 28, 2014 at 12:23 PM, Danijel Schiavuzzi < > [email protected]> wrote: > >> By default, the Kafka spout resumes consuming where it last left off. >> That offset is stored in ZooKeeper. >> >> You can set forceStartOffset to -2 to start consuming from the earliest >> available offset, or -1 to start consuming from the latest available offset. >> >> >> On Wednesday, May 28, 2014, Raphael Hsieh <[email protected]> wrote: >> >>> If I don't tell trident to start consuming data from the beginning of >>> the Kafka stream, where does it start from? >>> If I were to do: >>> tridentKafkaConfig.forceFromStart = true; >>> Then it will tell the spout to start consuming from the start of the >>> stream. If that is not set, then where does it start consuming from? and >>> How might I go about telling it to start consuming from the very end of the >>> stream? >>> >>> If a disaster were to happen and all my hosts died, when I start my >>> cluster back up, it might start consuming from where it left off. I would >>> rather manually process that old data, and have my storm system start >>> processing the live data. >>> >>> Thanks >>> -- >>> Raphael Hsieh >>> >>> >>> >>> >> >> >> -- >> Danijel Schiavuzzi >> >> E: [email protected] >> W: www.schiavuzzi.com >> T: +385989035562 >> Skype: danijels7 >> > > > > -- > Raphael Hsieh > Amazon.com > Software Development Engineer I > (978) 764-9014 > > > > -- Raphael Hsieh Amazon.com Software Development Engineer I (978) 764-9014
