Re: API for node entry to the cluster.
It is pretty easy to do by hand and is pretty easy to script (it was for us). The process for adding is a) configure and start a new node b) re-configure and restart each existing node in turn to know about the new node you now have a larger cluster. To drop a node, a) reconfigure and restart each surviving node b) kill the node(s) that is(are) leaving you now have a smaller cluster. On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: Hi All Is it possible to remove nodes and add nodes dynamically to the ZK cluster via API? Any plans in the future to do this? TIA A -- Ted Dunning, CTO DeepDyve
Re: API for node entry to the cluster.
I would prefer not restarting. Start/Stop the new/old process and then start a round of consensus for adding/removing a machine. I guess if one can do that then there is stopping of process required. Am I missing something here? A On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning ted.dunn...@gmail.com wrote: It is pretty easy to do by hand and is pretty easy to script (it was for us). The process for adding is a) configure and start a new node b) re-configure and restart each existing node in turn to know about the new node you now have a larger cluster. To drop a node, a) reconfigure and restart each surviving node b) kill the node(s) that is(are) leaving you now have a smaller cluster. On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: Hi All Is it possible to remove nodes and add nodes dynamically to the ZK cluster via API? Any plans in the future to do this? TIA A -- Ted Dunning, CTO DeepDyve
Re: API for node entry to the cluster.
You have two options. What Henry is talking about is a sophisticated update to ZK internals that does a very nice job of handling all the possible split-brain issues and other pathologies that can result from growing or shrinking a cluster. His patch isn't ready yet, but when it is, it should be really nice. What I am talking about is pretty tame hack that allows you to get 99% of the result needed with very little effort. It can be done today. Restarting servers one at a time is conventionally used for minor releases of ZK and can be (ab)used to allow configuration changes. I don't quite follow what you are saying in your second and third sentences so I can't answer the question in your fourth. Can you clarify what you mean? I halfway suspect you dropped a word somewhere. On Thu, Nov 5, 2009 at 11:24 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: I would prefer not restarting. Start/Stop the new/old process and then start a round of consensus for adding/removing a machine. I guess if one can do that then there is stopping of process required. Am I missing something here? A On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning ted.dunn...@gmail.com wrote: It is pretty easy to do by hand and is pretty easy to script (it was for us). The process for adding is a) configure and start a new node b) re-configure and restart each existing node in turn to know about the new node you now have a larger cluster. To drop a node, a) reconfigure and restart each surviving node b) kill the node(s) that is(are) leaving you now have a smaller cluster. On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: Hi All Is it possible to remove nodes and add nodes dynamically to the ZK cluster via API? Any plans in the future to do this? TIA A -- Ted Dunning, CTO DeepDyve -- Ted Dunning, CTO DeepDyve
Re: API for node entry to the cluster.
Ah. Sorry. But I got it. I guess what Henry is working on is what I am looking for. Thanks A On Thu, Nov 5, 2009 at 11:32 AM, Ted Dunning ted.dunn...@gmail.com wrote: You have two options. What Henry is talking about is a sophisticated update to ZK internals that does a very nice job of handling all the possible split-brain issues and other pathologies that can result from growing or shrinking a cluster. His patch isn't ready yet, but when it is, it should be really nice. What I am talking about is pretty tame hack that allows you to get 99% of the result needed with very little effort. It can be done today. Restarting servers one at a time is conventionally used for minor releases of ZK and can be (ab)used to allow configuration changes. I don't quite follow what you are saying in your second and third sentences so I can't answer the question in your fourth. Can you clarify what you mean? I halfway suspect you dropped a word somewhere. On Thu, Nov 5, 2009 at 11:24 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: I would prefer not restarting. Start/Stop the new/old process and then start a round of consensus for adding/removing a machine. I guess if one can do that then there is stopping of process required. Am I missing something here? A On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning ted.dunn...@gmail.com wrote: It is pretty easy to do by hand and is pretty easy to script (it was for us). The process for adding is a) configure and start a new node b) re-configure and restart each existing node in turn to know about the new node you now have a larger cluster. To drop a node, a) reconfigure and restart each surviving node b) kill the node(s) that is(are) leaving you now have a smaller cluster. On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: Hi All Is it possible to remove nodes and add nodes dynamically to the ZK cluster via API? Any plans in the future to do this? TIA A -- Ted Dunning, CTO DeepDyve -- Ted Dunning, CTO DeepDyve
Re: API for node entry to the cluster.
Hi - Yes there are future plans. See https://issues.apache.org/jira/browse/ZOOKEEPER-107. I have code written for this that works but is not rock-solid yet. cheers, Henry On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman avinash.laksh...@gmail.com wrote: Hi All Is it possible to remove nodes and add nodes dynamically to the ZK cluster via API? Any plans in the future to do this? TIA A
API for node entry to the cluster.
Hi All Is it possible to remove nodes and add nodes dynamically to the ZK cluster via API? Any plans in the future to do this? TIA A
ZooKeeper talks at the post-Apachecon Hadoop Meetup tonight
Apologies for the late notice, but I wanted to advertise a pair of short talks by Mahadev and myself at the Hadoop Meetup tonight in Oakland: Mahadev will be giving a broad overview of ZooKeeper and talking about its uses inside and outside of Yahoo!, and I will be talking about upcoming features that I've been working on including observers and dynamic ensembles. There will be other talks as well on HDFS and other topics. Rumour has it there will be beer (but don't hold me to that!). The meetup is at the Mariott Oakland City Center. Google map is here: http://maps.google.com/maps?oe=utf-8client=firefox-aie=UTF8q=merriot+oaklandfb=1gl=ushq=merriothnear=oaklandcid=0,0,204705793290918968ei=k3jzSqbJA4WksgPZ58gdved=0CAwQnwIwAAll=37.803087,-122.272575spn=0.009766,0.022724z=16iwloc=A I'll post my slides in the next day or so. Hope some of you can make it! cheers, Henry Henry Robinson Software Engineer Cloudera
