Re: Some questions about Zookeeper 3.2.0
Thanks Mahadev, your information is really helpful :-) On Wed, Jul 1, 2009 at 12:14 AM, Mahadev Konar wrote: > Hi Qian, > Sorry for the delayed response. ZooKeeper guarantees that their is > backwards compatibility with minor releases. So, 3.2* is backwards > compatible with 3.1*. So, what Ted mentions below related to rolling > upgrade > should work. > > Hope this helps. > Thanks > Mahadev > > > Ted Dunning: > > A rolling update works very well for that. You can also change the number > of nodes in the cluster. > > To do this, you replace the config files on the surviving servers and on > the > new server. > > Then take down the one that is leaving the cluster and then one by one > restart the servers that will remain in the cluster. Then start the new > server. > > If you only have 3 ZK servers, you might want to do the upgrade in two > steps > if you are totally paranoid about running with just 2 servers in your > cluster. In that scenario, you would add the new server to the config and > restart each of the 3 ZK servers and staring the new one. That gives you a > ZK cluster with 4 servers. Another rolling update can reconfigure it to > have only the 3 servers you want. > > Make sure you leave several seconds between restarting each server. That > will give the cluster time to calm down and do any necessary leader > elections > > > On 6/28/09 11:03 PM, "Qian Ye" wrote: > > > A related question. If I have run a Zookeeper service with many > ephemeral > > znodes on five servers, then I want to replace one of the servers by a > new > > one, and the new server has a new IP. How can I do the job without any > loss > > of data ? > > > > Thanks > > > > > > On Mon, Jun 29, 2009 at 9:24 AM, Qian Ye wrote: > > > >> Thanks for the explanation, it sounds great. > >> > >> It seems that I should study more on Zookeeper :-) > >> > >> regards > >> > >> > >> On Mon, Jun 29, 2009 at 2:15 AM, Ted Dunning >wrote: > >> > >>> I don't think you should be very nervous at all. > >>> > >>> There are two questions: > >>> > >>> 1) can 3.1.1 go to 3.2 with no down time. This is very likely, but a > >>> wiser > >>> head than mine should have final say > >>> > >>> 2) can 3.1.1 go to 3.2 with < 1 minute of downtime. The is for sure. > >>> > >>> Neither option involves data loss. > >>> > >>> ZK is actually really really good for HA operations. > >>> > >>> On Sun, Jun 28, 2009 at 4:55 AM, Qian Ye wrote: > >>> > your answer makes me nervous about upgrade Zookeeper > server > > >>> > >> > >> > >> > >> -- > >> With Regards! > >> > >> Ye, Qian > >> Made in Zhejiang University > >> > >> > > > > -- With Regards! Ye, Qian Made in Zhejiang University
Re: Some questions about Zookeeper 3.2.0
Hi Qian, Sorry for the delayed response. ZooKeeper guarantees that their is backwards compatibility with minor releases. So, 3.2* is backwards compatible with 3.1*. So, what Ted mentions below related to rolling upgrade should work. Hope this helps. Thanks Mahadev Ted Dunning: A rolling update works very well for that. You can also change the number of nodes in the cluster. To do this, you replace the config files on the surviving servers and on the new server. Then take down the one that is leaving the cluster and then one by one restart the servers that will remain in the cluster. Then start the new server. If you only have 3 ZK servers, you might want to do the upgrade in two steps if you are totally paranoid about running with just 2 servers in your cluster. In that scenario, you would add the new server to the config and restart each of the 3 ZK servers and staring the new one. That gives you a ZK cluster with 4 servers. Another rolling update can reconfigure it to have only the 3 servers you want. Make sure you leave several seconds between restarting each server. That will give the cluster time to calm down and do any necessary leader elections On 6/28/09 11:03 PM, "Qian Ye" wrote: > A related question. If I have run a Zookeeper service with many ephemeral > znodes on five servers, then I want to replace one of the servers by a new > one, and the new server has a new IP. How can I do the job without any loss > of data ? > > Thanks > > > On Mon, Jun 29, 2009 at 9:24 AM, Qian Ye wrote: > >> Thanks for the explanation, it sounds great. >> >> It seems that I should study more on Zookeeper :-) >> >> regards >> >> >> On Mon, Jun 29, 2009 at 2:15 AM, Ted Dunning wrote: >> >>> I don't think you should be very nervous at all. >>> >>> There are two questions: >>> >>> 1) can 3.1.1 go to 3.2 with no down time. This is very likely, but a >>> wiser >>> head than mine should have final say >>> >>> 2) can 3.1.1 go to 3.2 with < 1 minute of downtime. The is for sure. >>> >>> Neither option involves data loss. >>> >>> ZK is actually really really good for HA operations. >>> >>> On Sun, Jun 28, 2009 at 4:55 AM, Qian Ye wrote: >>> your answer makes me nervous about upgrade Zookeeper server >>> >> >> >> >> -- >> With Regards! >> >> Ye, Qian >> Made in Zhejiang University >> >> >
Re: Some questions about Zookeeper 3.2.0
A rolling update works very well for that. You can also change the number of nodes in the cluster. To do this, you replace the config files on the surviving servers and on the new server. Then take down the one that is leaving the cluster and then one by one restart the servers that will remain in the cluster. Then start the new server. If you only have 3 ZK servers, you might want to do the upgrade in two steps if you are totally paranoid about running with just 2 servers in your cluster. In that scenario, you would add the new server to the config and restart each of the 3 ZK servers and staring the new one. That gives you a ZK cluster with 4 servers. Another rolling update can reconfigure it to have only the 3 servers you want. Make sure you leave several seconds between restarting each server. That will give the cluster time to calm down and do any necessary leader elections On Sun, Jun 28, 2009 at 11:03 PM, Qian Ye wrote: > A related question. If I have run a Zookeeper service with many ephemeral > znodes on five servers, then I want to replace one of the servers by a new > one, and the new server has a new IP. How can I do the job without any loss > of data ? > >
Re: Some questions about Zookeeper 3.2.0
A related question. If I have run a Zookeeper service with many ephemeral znodes on five servers, then I want to replace one of the servers by a new one, and the new server has a new IP. How can I do the job without any loss of data ? Thanks On Mon, Jun 29, 2009 at 9:24 AM, Qian Ye wrote: > Thanks for the explanation, it sounds great. > > It seems that I should study more on Zookeeper :-) > > regards > > > On Mon, Jun 29, 2009 at 2:15 AM, Ted Dunning wrote: > >> I don't think you should be very nervous at all. >> >> There are two questions: >> >> 1) can 3.1.1 go to 3.2 with no down time. This is very likely, but a >> wiser >> head than mine should have final say >> >> 2) can 3.1.1 go to 3.2 with < 1 minute of downtime. The is for sure. >> >> Neither option involves data loss. >> >> ZK is actually really really good for HA operations. >> >> On Sun, Jun 28, 2009 at 4:55 AM, Qian Ye wrote: >> >> > your answer makes me nervous about upgrade Zookeeper >> > server >> > >> > > > > -- > With Regards! > > Ye, Qian > Made in Zhejiang University > > -- With Regards! Ye, Qian Made in Zhejiang University
Re: Some questions about Zookeeper 3.2.0
Thanks for the explanation, it sounds great. It seems that I should study more on Zookeeper :-) regards On Mon, Jun 29, 2009 at 2:15 AM, Ted Dunning wrote: > I don't think you should be very nervous at all. > > There are two questions: > > 1) can 3.1.1 go to 3.2 with no down time. This is very likely, but a wiser > head than mine should have final say > > 2) can 3.1.1 go to 3.2 with < 1 minute of downtime. The is for sure. > > Neither option involves data loss. > > ZK is actually really really good for HA operations. > > On Sun, Jun 28, 2009 at 4:55 AM, Qian Ye wrote: > > > your answer makes me nervous about upgrade Zookeeper > > server > > > -- With Regards! Ye, Qian Made in Zhejiang University
Re: Some questions about Zookeeper 3.2.0
I don't think you should be very nervous at all. There are two questions: 1) can 3.1.1 go to 3.2 with no down time. This is very likely, but a wiser head than mine should have final say 2) can 3.1.1 go to 3.2 with < 1 minute of downtime. The is for sure. Neither option involves data loss. ZK is actually really really good for HA operations. On Sun, Jun 28, 2009 at 4:55 AM, Qian Ye wrote: > your answer makes me nervous about upgrade Zookeeper > server >
Re: Some questions about Zookeeper 3.2.0
Thanks Ted, frankly speaking, your answer makes me nervous about upgrade Zookeeper server. Hope the developers of Zookeeper thinking about it. I think Zookeeper is very cool. But it isn't good enough for online systems if it cannot be upgraded without losing existed data. Thanks On Sat, Jun 27, 2009 at 3:13 PM, Ted Dunning wrote: > In general for changes like this, you need to be running more than one > server in a cluster to avoid losing state such as the ephemeral nodes. > > I can't say for certain that the 3.1.1 to 3.2 change can be done this way, > but most upgrades can be done by stopping one server at a time, changing > the software and config and restarting it. Then you move on to the next. > This will cause a connection loss event for each client eventually, but it > should > not cause any sessions to expire. > > Don't do this until somebody says something for certain that 3.1.1 can > change to 3.2 within a cluster. > > On Fri, Jun 26, 2009 at 7:16 PM, Qian Ye wrote: > > > 2. How to upgrade from 3.1.1 to 3.2.0 properly. Just stop the old server > > and > > start the new server would be OK? Will this operation influence the old > > ephemeral znodes? Is there a perfect upgrade plan? > > > -- With Regards! Ye, Qian Made in Zhejiang University
Re: Some questions about Zookeeper 3.2.0
In general for changes like this, you need to be running more than one server in a cluster to avoid losing state such as the ephemeral nodes. I can't say for certain that the 3.1.1 to 3.2 change can be done this way, but most upgrades can be done by stopping one server at a time, changing the software and config and restarting it. Then you move on to the next. This will cause a connection loss event for each client eventually, but it should not cause any sessions to expire. Don't do this until somebody says something for certain that 3.1.1 can change to 3.2 within a cluster. On Fri, Jun 26, 2009 at 7:16 PM, Qian Ye wrote: > 2. How to upgrade from 3.1.1 to 3.2.0 properly. Just stop the old server > and > start the new server would be OK? Will this operation influence the old > ephemeral znodes? Is there a perfect upgrade plan? >