Well, I am running zookeeper on EC2 and rely on configuration management to update configuration files when topology changes. Chef and Pallet in particular are very good at addressing that type of problem. As for haproxy shifting the problem, you can rely on elastic ips and two load-balancing instances. If you want to keep the moving parts and types of nodes down and don't already run a load-balancing service, the CM approach is your best bet
On Fri, Jan 6, 2012 at 11:04 AM, Tobias Schulze-Heinrichs <t.schu...@xplosion.de> wrote: > Hmm, that shift the problem just to another resource. If the haproxy instance > crashes I'll need to reconfigure the producer and consumers. I guess that > problem needs to be addressed to the experts of cloud computing. > > Beside that I read that the zookeeper guys just working on a way to > dynamically add and/or remove zookeeper servers to an existing ensemble. > > Best regards > Tobias > > > Am 06.01.2012 um 09:18 schrieb Pierre-Yves Ritschard: > >> or you could load-balance access to your ensemble using haproxy or elb >> >> On Fri, Jan 6, 2012 at 9:06 AM, Tobias Schulze-Heinrichs >> <t.schu...@xplosion.de> wrote: >>> Hi Neha, >>> >>> is this an issue which is not implemented in Kafka or does zookeeper >>> doesn't offer this information? To describe this problem a little bit more >>> detailed - we want to set up a service at the cloud (amazon ec2). Therefor >>> it might occur that one instance crashes and came up with a new IP. In that >>> scenario I have to reconfigure and restart all Zookeeper and all Kafka >>> server, right? >>> >>> Best regards >>> Tobias >>> >>> >>> Am 05.01.2012 um 21:39 schrieb Neha Narkhede: >>> >>>> Tobias, >>>> >>>> Please configure Kafka with the list of all 3 zookeeper servers. The >>>> way the zookeeper client works is it just picks one zookeeper server >>>> from the list of all zookeeper servers in the connection url. If you >>>> miss specifying one server in your connection url, Kafka will never >>>> connect to the 3rd server, causing imbalance on the zookeeper cluster. >>>> >>>> Thanks, >>>> Neha >>>> >>>> On Thu, Jan 5, 2012 at 12:34 PM, Tobias Schulze-Heinrichs >>>> <t.schu...@xplosion.de> wrote: >>>>> Hello all, >>>>> >>>>> I have an ensemble of 3 ZK Server. Do I need to configure all 3 zookeeper >>>>> servers to my Kafka server or is there a possibility that it can get the >>>>> address of 2 ZK out of one which is given at my Kafka configuration? >>>>> >>>>> Best regards >>>>> Tobias >>> >
