Re: create topic in multiple node kafka cluster

2014-10-09 Thread Sa Li
Hi, I kinda doubt whether I make it as an ensemble, since it shows root@DO-mq-dev:/etc/zookeeper/conf# zkServer.sh status JMX enabled by default Using config: /etc/zookeeper/conf/zoo.cfg Mode: standalone Mode is standalone instead of something else, here is my zoo.cfg, I did follow the instructi

Re: create topic in multiple node kafka cluster

2014-10-09 Thread Guozhang Wang
Sa, Usually you would not want to set up kafka brokers at the same machines with zk nodes, as that will add depending failures to the server cluster. Back to your original question, it seems your zk nodes do not form an ensemble, since otherwise their zk data should be the same. Guozhang On Thu

Re: create topic in multiple node kafka cluster

2014-10-09 Thread Joel Koshy
It looks like You set up three separate ZK clusters, not an ensemble. You can take a look at http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_zkMulitServerSetup on how to set up an ensemble; and then register all three kafka brokers on that single zk ensemble. Joel On Thu, Oct 09, 2

create topic in multiple node kafka cluster

2014-10-09 Thread Sa Li
Hi, All I setup a 3-node kafka cluster on top of 3-node zk ensemble. Now I launch 1 broker on each node, the brokers will be randomly distributed to zk ensemble, see DO-mq-dev.1 [zk: localhost:2181(CONNECTED) 1] ls /brokers/ids [0, 1] pof-kstorm-dev1.2 [zk: localhost:2181(CONNECTED) 1] ls /broke