hmmm - and if you turn off zookeeper? On Thu, Nov 17, 2011 at 9:15 PM, Inder Pall <inder.p...@gmail.com> wrote:
> The consumer offsets are stored in ZooKeeper by topic and partition. > That's how in a consumer fail over scenario you don't get duplicate > messages > > - Inder > > On Fri, Nov 18, 2011 at 10:33 AM, Taylor Gautier <tgaut...@tagged.com > >wrote: > > > Hi, > > > > We've noticed that the cleaner script in Kafka removes empty log segments > > but not the directories themselves. I am actually wondering something - > I > > always assumed that Kafka could restore the latest offset for existing > > topics by scanning the log directory for all directories and scanning the > > directories for log segment files to restore the latest offset. > > > > Now this conclusion I have made simply by observation - so it could be > > entirely wrong. > > > > My question is however - if I am right, and the cleaner removes all the > log > > segments for a given topic so that a given topic directory is empty, how > > does Kafka behave when restarted? How does it know what the next offset > > should be? > > > > > > -- > -- Inder >
