Thank you very much, After i created a static path for ZKRoot and spoutConfig.forceFromStart set to false now reading last offset and consuming unread messages.
On Tue, Mar 10, 2015 at 11:53 AM, Harsha <[email protected]> wrote: > Yes thats bad approach . Mostly users keep a static string for the "id" > part in the spoutConfig. Whats the need to use randomUUID. > > -- > Harsha > > > On March 9, 2015 at 11:09:46 PM, Tousif ([email protected]) wrote: > > Thanks Harsha, > > Does zkRoot in the spoutconfig is used along with random string to > create path in Zookeeper? > > SpoutConfig(hosts, PropertyManager.getProperty(" > kafka.spout.topic").toString(), "/kafkastormroot", > *UUID.randomUUID().toString()*); > > zkroot is prepended to the random id we give to create the unique place > in ZK where consumption state is stored. If this is true than every time > we deploy our topology it will create a unique path ? > > How will it get previous offset? > > > On Mon, Mar 9, 2015 at 8:36 PM, Harsha <[email protected]> wrote: > >> If your topology has saved Kafka offset in your zookeeper it will >> start processing from that otherwise It checks spoutConfig.forceFromStart >> set to true in this case it will try to fetch data from the beginning of >> the queue i.e kafka.api.EarliestTime() . If none of the above matches it >> will pick users spoutConfig.startOffsetTime. >> >> "kafka.api.OffsetRequest.EarliestTime() finds the beginning of the data >> in the logs and starts streaming from there, >> kafka.api.OffsetRequest.LatestTime() will only stream new messages." >> >> If you stopping and re-deploying the topology make sure you used the same >> name as KafkaSpout uses topology name to store and retrieve the offsets >> from zookeeper. >> >> -- >> Harsha >> >> >> >> On March 9, 2015 at 7:30:38 AM, Tousif ([email protected]) wrote: >> >> If your topology has saved Kafka offset in your zookeeper it will start >> processing from that otherwise It checks spoutConfig.forceFromStart set to >> true in this case it will try to fetch data from the beginning of the queue >> i.e kafka.api.EarliestTime() . If none of the above matches it will pick >> users spoutConfig.startOffsetTime. >> >> "kafka.api.OffsetRequest.EarliestTime() finds the beginning of the data >> in the logs and starts streaming from there, >> kafka.api.OffsetRequest.LatestTime() will only stream new messages." >> >> If you stopping and re-deploying the topology make sure you used the >> same name as KafkaSpout uses topology name to store and retrieve the >> offsets from zookeeper. >> >> -- >> Harsha >> >> > > > -- > > > Regards > Tousif Khazi > > -- Regards Tousif Khazi
