Okay, thanks for clarifying. And this includes 0.7.0 as well?
On Fri, Oct 5, 2012 at 11:05 AM, Jun Rao <jun...@gmail.com> wrote:
> Mark,
>
> ZK-based producer is intended to support adding brokers dynamically.
> However, because of a bug, in 0.7.x, you will need to make sure that a
> topic already exists on a new broker.
>
> Thanks,
>
> Jun
>
> On Thu, Oct 4, 2012 at 10:17 AM, Mark Grabois <mark.grab...@trendrr.com
> >wrote:
>
> > Okay, I was able to produce messages directly to a broker which was part
> of
> > the quorum list in zookeeper.properties. Then, once I restart the
> ZK-based
> > producer, it successfully produces to that broker. However, I can't seem
> to
> > add a kafka broker to a quorum if no topics are registered on it.
> >
> > Should I be able to add a kafka node to a zk-based quorum and produce
> > messages to it immediately, without first producing directly to that node
> > and then restarting ZK-based producer?
> >
> >
> > On Wed, Oct 3, 2012 at 12:14 PM, Jun Rao <jun...@gmail.com> wrote:
> >
> > > Mark,
> > >
> > > If you want to use all brokers, you need to specify all brokers in
> > > broker.list? You may need to make sure that the topic is already
> created
> > on
> > > each broker first (by sending at least one message to it directly).
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Oct 2, 2012 at 8:45 AM, Mark Grabois <mark.grab...@trendrr.com
> > > >wrote:
> > >
> > > > Hi Jun, sorry for the delay, I'd like to revisit the issue I was
> having
> > > > earlier
> > > >
> > > > using the Kafka Java API, I initialized a KafkaProducer with
> following
> > > > parameters:
> > > >
> > > > props.put("serializer.class", "kafka.serializer.StringEncoder");
> > > > props.put("producer.type", "async");
> > > > props.put("compression.codec", "1");
> > > >
> > > > props.put("broker.list", "0:<host_ip>:9092"); //instead of zk.connect
> > > > string
> > > > producer = new Producer<Integer, String>(new ProducerConfig(props));
> > > > producer.send(new ProducerData<Integer, String>(topicId, message));
> > > > -- where topicId is a new topic, not an existing one
> > > >
> > > > this did not seem to produce a message to the third node, which is
> not
> > > > listening to my producer on 9092 (or any other port)
> > > >
> > > > However, I also tried to produce to my active brokers which are
> > correctly
> > > > receiving messages from other producers. This didn't work by just
> > > > specifying the host ip in the broker.list param of a newly
> initialized
> > > > Producer. Is there something else I need to configure for this to
> work?
> > > >
> > > > Thank you,
> > > > Mark
> > > >
> > > > On Wed, Sep 12, 2012 at 12:18 AM, Jun Rao <jun...@gmail.com> wrote:
> > > >
> > > > > This may be due to a bug in 0.7.1 with respect to ZK based
> producer.
> > > > Could
> > > > > you try sending 1 produce request to the new broker directly first
> > and
> > > > then
> > > > > try the ZK based producer?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Sep 11, 2012 at 9:41 AM, Mark Grabois <
> > > mark.grab...@trendrr.com
> > > > > >wrote:
> > > > >
> > > > > > I think it's a broker issue. The producers are connected to the
> > > > zookeeper
> > > > > > running on the node, and currently they are producing messages
> > > > correctly
> > > > > to
> > > > > > the other nodes.
> > > > > >
> > > > > > On Mon, Sep 10, 2012 at 11:32 PM, Jun Rao <jun...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > Mark,
> > > > > > >
> > > > > > > Is this a broker side issue or a producer side issue?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Mon, Sep 10, 2012 at 9:23 AM, Mark Grabois <
> > > > > mark.grab...@trendrr.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > > Currently I have two nodes with both zookeeper and kafka
> > > instances
> > > > > > > running
> > > > > > > > successfully, processing messages with connections to both
> > > producer
> > > > > and
> > > > > > > > consumer clients. However, when I try to add a third kafka+zk
> > > node
> > > > to
> > > > > > the
> > > > > > > > quorum, configured in the exact same way and having specified
> > all
> > > > > > broker
> > > > > > > > ids and server ips properly, kafka is unable to establish a
> > > > > connection
> > > > > > to
> > > > > > > > my producers on the default listening port (9092). Zookeeper
> > > > appears
> > > > > to
> > > > > > > be
> > > > > > > > connecting properly to the cluster, but not Kafka.
> > > > > > > >
> > > > > > > > Any recommendations on how to troubleshoot this? All nodes
> are
> > > > > running
> > > > > > > > kafka-0.7.1.
> > > > > > > >
> > > > > > > > Thank you,
> > > > > > > > Mark
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
