Yes, this bug exists in 0.7.0 too. The issue should be fixed in 0.8.
Thanks,
Jun
On Fri, Oct 5, 2012 at 8:41 AM, Mark Grabois <mark.grab...@trendrr.com>wrote:
> Okay, thanks for clarifying. And this includes 0.7.0 as well?
>
> On Fri, Oct 5, 2012 at 11:05 AM, Jun Rao <jun...@gmail.com> wrote:
>
> > Mark,
> >
> > ZK-based producer is intended to support adding brokers dynamically.
> > However, because of a bug, in 0.7.x, you will need to make sure that a
> > topic already exists on a new broker.
> >
> > Thanks,
> >
> > Jun
> >
> > On Thu, Oct 4, 2012 at 10:17 AM, Mark Grabois <mark.grab...@trendrr.com
> > >wrote:
> >
> > > Okay, I was able to produce messages directly to a broker which was
> part
> > of
> > > the quorum list in zookeeper.properties. Then, once I restart the
> > ZK-based
> > > producer, it successfully produces to that broker. However, I can't
> seem
> > to
> > > add a kafka broker to a quorum if no topics are registered on it.
> > >
> > > Should I be able to add a kafka node to a zk-based quorum and produce
> > > messages to it immediately, without first producing directly to that
> node
> > > and then restarting ZK-based producer?
> > >
> > >
> > > On Wed, Oct 3, 2012 at 12:14 PM, Jun Rao <jun...@gmail.com> wrote:
> > >
> > > > Mark,
> > > >
> > > > If you want to use all brokers, you need to specify all brokers in
> > > > broker.list? You may need to make sure that the topic is already
> > created
> > > on
> > > > each broker first (by sending at least one message to it directly).
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Oct 2, 2012 at 8:45 AM, Mark Grabois <
> mark.grab...@trendrr.com
> > > > >wrote:
> > > >
> > > > > Hi Jun, sorry for the delay, I'd like to revisit the issue I was
> > having
> > > > > earlier
> > > > >
> > > > > using the Kafka Java API, I initialized a KafkaProducer with
> > following
> > > > > parameters:
> > > > >
> > > > > props.put("serializer.class", "kafka.serializer.StringEncoder");
> > > > > props.put("producer.type", "async");
> > > > > props.put("compression.codec", "1");
> > > > >
> > > > > props.put("broker.list", "0:<host_ip>:9092"); //instead of
> zk.connect
> > > > > string
> > > > > producer = new Producer<Integer, String>(new
> ProducerConfig(props));
> > > > > producer.send(new ProducerData<Integer, String>(topicId, message));
> > > > > -- where topicId is a new topic, not an existing one
> > > > >
> > > > > this did not seem to produce a message to the third node, which is
> > not
> > > > > listening to my producer on 9092 (or any other port)
> > > > >
> > > > > However, I also tried to produce to my active brokers which are
> > > correctly
> > > > > receiving messages from other producers. This didn't work by just
> > > > > specifying the host ip in the broker.list param of a newly
> > initialized
> > > > > Producer. Is there something else I need to configure for this to
> > work?
> > > > >
> > > > > Thank you,
> > > > > Mark
> > > > >
> > > > > On Wed, Sep 12, 2012 at 12:18 AM, Jun Rao <jun...@gmail.com>
> wrote:
> > > > >
> > > > > > This may be due to a bug in 0.7.1 with respect to ZK based
> > producer.
> > > > > Could
> > > > > > you try sending 1 produce request to the new broker directly
> first
> > > and
> > > > > then
> > > > > > try the ZK based producer?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Tue, Sep 11, 2012 at 9:41 AM, Mark Grabois <
> > > > mark.grab...@trendrr.com
> > > > > > >wrote:
> > > > > >
> > > > > > > I think it's a broker issue. The producers are connected to the
> > > > > zookeeper
> > > > > > > running on the node, and currently they are producing messages
> > > > > correctly
> > > > > > to
> > > > > > > the other nodes.
> > > > > > >
> > > > > > > On Mon, Sep 10, 2012 at 11:32 PM, Jun Rao <jun...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > Mark,
> > > > > > > >
> > > > > > > > Is this a broker side issue or a producer side issue?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Mon, Sep 10, 2012 at 9:23 AM, Mark Grabois <
> > > > > > mark.grab...@trendrr.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > Hi All,
> > > > > > > > >
> > > > > > > > > Currently I have two nodes with both zookeeper and kafka
> > > > instances
> > > > > > > > running
> > > > > > > > > successfully, processing messages with connections to both
> > > > producer
> > > > > > and
> > > > > > > > > consumer clients. However, when I try to add a third
> kafka+zk
> > > > node
> > > > > to
> > > > > > > the
> > > > > > > > > quorum, configured in the exact same way and having
> specified
> > > all
> > > > > > > broker
> > > > > > > > > ids and server ips properly, kafka is unable to establish a
> > > > > > connection
> > > > > > > to
> > > > > > > > > my producers on the default listening port (9092).
> Zookeeper
> > > > > appears
> > > > > > to
> > > > > > > > be
> > > > > > > > > connecting properly to the cluster, but not Kafka.
> > > > > > > > >
> > > > > > > > > Any recommendations on how to troubleshoot this? All nodes
> > are
> > > > > > running
> > > > > > > > > kafka-0.7.1.
> > > > > > > > >
> > > > > > > > > Thank you,
> > > > > > > > > Mark
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
