Thanks everyone for the response. To summarize, config.forceFromStart lets kafka spout start consuming from the topic's beginning (the earliest offset available for consumption) and config.startOffsetTime =earliestTime() is the offset where spout left off consuming last time.
On Mon, Mar 30, 2015 at 3:03 PM, P. Taylor Goetz <[email protected]> wrote: > The naming of config.forceFromStart is a little confusing, but in a > nutshell if true, it enables the config.startOffsetTime setting. > > If false, the spout will look for an offset in ZK (if any) and start from > there. If there's no offset in ZK, it will start from the end. > > -Taylor > > > On Mar 30, 2015, at 5:35 PM, Sarang Deshpande <[email protected]> > wrote: > > Hello storm users, > There are two variables in the Kafka spout config, startOffsetTime and > forceFromStart. If I set these as follows - > > config.forceFromStart = *false*; > config.startOffsetTime = OffsetRequest.EarliestTime(); > > Kafka spout reads only the newly published messages (from latest offset) > and DOES NOT read from start as one would expect. Only when config is set > as follows, Kafka spout reads from the beginning. > > config.forceFromStart = *true*; > config.startOffsetTime = OffsetRequest.EarliestTime(); > > Can someone explain what is the logical difference between forceFromStart > and startOffsetTime being EarliestTime? > > Thanks much > ~Sarang > > PS: Note that on the other hand, if forceFromStart is true and > startOffsetTime is LatestTime(), that is if they contradict with each > other, then spout reads only newly published messages. > >
