What Guozhang said is essentially what I was saying. I realized my code is not directly applicable to your problem as it only checks the partition offsets and not whether a consumer is actually consuming the partitions.
--Ian On Mar 20, 2014, at 5:38 PM, Todd Gatts <tga...@us.ibm.com> wrote: > Guozhang: > > Nice approach. I'll give that a try as well. Thanks. > > Todd > > > Guozhang Wang ---2014-03-19 07:02:20 PM---Currently producers cannot detect > if a topic is consumed by someone or not themselves. If you want t > > From: Guozhang Wang <wangg...@gmail.com> > To: "users@kafka.apache.org" <users@kafka.apache.org>, > Date: 2014-03-19 07:02 PM > Subject: Re: Can a producer detect when a topic has no consumers? > > > > Currently producers cannot detect if a topic is consumed by someone or not > themselves. If you want to know globally if a specific topic is consumed by > anyone or not, you can query the ZK for all the consumer groups, and search > if they contain the topic name. > > Guozhang > > > On Wed, Mar 19, 2014 at 1:01 PM, Ian Friedman <i...@flurry.com> wrote: > > > Hey Todd, > > > > For my purposes, I wrote a Partitioner that reads all of the > > ConsumerOffsets for a particular group from Zookeeper, and then sends its > > message to the one with least lag (with some probabilistic weighting thrown > > in to prevent it from getting overwhelmed). I basically read and > > reimplemented the code from ConsumerOffsetChecker.scala to do this. Does > > that help at all? Would it help to see my code? > > > > --Ian > > > > On Mar 19, 2014, at 12:47 PM, Todd Gatts <tga...@us.ibm.com> wrote: > > > > > > > > > > > I'd like to avoid creating messages for a topic is the topic currently > > has > > > no consumers. Is there a way a producer can subscribe to the number of > > > consumers of a topic? Or if that's not possible, can a producer request > > > the current number of consumers of a topic? > > > > > > The searchable hadoop kafka topic didn't return anything for "no > > consumer" > > > or "zero consumers", and a find on "zero" or "count" on the documentation > > > html page found lots of matches, but none about knowing the number of > > > consumers. > > > > > > I'm certain I can code a solution around this problem, but I'd hate to > > > reinvent the wheel for such a straightforward problem. > > > > > > Todd Gatts > > > > > > > -- > -- Guozhang >
