RE: Re: ephemerals handling after restart
Yes Ben -Original Message- From: Johannes Zillmann [mailto:[EMAIL PROTECTED] Sent: Thursday, September 18, 2008 02:22 AM Pacific Standard Time To: zookeeper-user@hadoop.apache.org Subject:Re: ephemerals handling after restart Hi Ben, thanks for your answer! Is the session recoverable in case the zk server was restarted in meantime ? Johannes On Sep 12, 2008, at 3:52 PM, Benjamin Reed wrote: > If a application does not close the ZooKeeper session before shutting > down, ZooKeeper will not cleanup the session until it times out. So > when > an application crashes and restarts, ZooKeeper doesn't know if the > client is a restart of an old client or a new client. > > There is a way to alleviate this problem: you can actually maintain a > session across client application restarts. If you save off the > session > id and password, when you restart you can try to reconnect to the > session using the ZooKeeper constructor that takes the old session id > and password. If the reconnect is successful you can then close the > session and get everything to clean up immediately. (Or you could keep > using the recovered session if you want to.) > > ben > > -Original Message- > From: Johannes Zillmann [mailto:[EMAIL PROTECTED] > Sent: Friday, September 12, 2008 2:49 AM > To: zookeeper-user@hadoop.apache.org > Subject: ephemerals handling after restart > > Hi all, > > i have a question regarding ephemerals and it behavior on client > crash/ > restart. > We've a master/node cluster similar to a hadoop hdfs cluster but using > zk for management. > The nodes creates an ephemeral to announce there existence to the > master. > > Now what i recognized is that after stopping the whole cluster and > starting only the master again, still some ephemeral nodes may exists > for some seconds. > That leads me to following questions. > What is if a node starts up again. Do it have to clean up it "old" > ephemeral node, or can it somehow acquire the old one ? > > Just trying to find a best way how to deal with this, since on regular > restart of the cluster i often recognize something like this > master : node1, node2 connected > master : node2 disconnected > master : node2 connected again > > Thanks for any help > Johannes > > ~~~ > 101tec GmbH > Halle (Saale), Saxony-Anhalt, Germany > http://www.101tec.com > > ~~~ 101tec GmbH Halle (Saale), Saxony-Anhalt, Germany http://www.101tec.com
Re: ephemerals handling after restart
Hi Ben, thanks for your answer! Is the session recoverable in case the zk server was restarted in meantime ? Johannes On Sep 12, 2008, at 3:52 PM, Benjamin Reed wrote: If a application does not close the ZooKeeper session before shutting down, ZooKeeper will not cleanup the session until it times out. So when an application crashes and restarts, ZooKeeper doesn't know if the client is a restart of an old client or a new client. There is a way to alleviate this problem: you can actually maintain a session across client application restarts. If you save off the session id and password, when you restart you can try to reconnect to the session using the ZooKeeper constructor that takes the old session id and password. If the reconnect is successful you can then close the session and get everything to clean up immediately. (Or you could keep using the recovered session if you want to.) ben -Original Message- From: Johannes Zillmann [mailto:[EMAIL PROTECTED] Sent: Friday, September 12, 2008 2:49 AM To: zookeeper-user@hadoop.apache.org Subject: ephemerals handling after restart Hi all, i have a question regarding ephemerals and it behavior on client crash/ restart. We've a master/node cluster similar to a hadoop hdfs cluster but using zk for management. The nodes creates an ephemeral to announce there existence to the master. Now what i recognized is that after stopping the whole cluster and starting only the master again, still some ephemeral nodes may exists for some seconds. That leads me to following questions. What is if a node starts up again. Do it have to clean up it "old" ephemeral node, or can it somehow acquire the old one ? Just trying to find a best way how to deal with this, since on regular restart of the cluster i often recognize something like this master : node1, node2 connected master : node2 disconnected master : node2 connected again Thanks for any help Johannes ~~~ 101tec GmbH Halle (Saale), Saxony-Anhalt, Germany http://www.101tec.com ~~~ 101tec GmbH Halle (Saale), Saxony-Anhalt, Germany http://www.101tec.com
RE: ephemerals handling after restart
If a application does not close the ZooKeeper session before shutting down, ZooKeeper will not cleanup the session until it times out. So when an application crashes and restarts, ZooKeeper doesn't know if the client is a restart of an old client or a new client. There is a way to alleviate this problem: you can actually maintain a session across client application restarts. If you save off the session id and password, when you restart you can try to reconnect to the session using the ZooKeeper constructor that takes the old session id and password. If the reconnect is successful you can then close the session and get everything to clean up immediately. (Or you could keep using the recovered session if you want to.) ben -Original Message- From: Johannes Zillmann [mailto:[EMAIL PROTECTED] Sent: Friday, September 12, 2008 2:49 AM To: zookeeper-user@hadoop.apache.org Subject: ephemerals handling after restart Hi all, i have a question regarding ephemerals and it behavior on client crash/ restart. We've a master/node cluster similar to a hadoop hdfs cluster but using zk for management. The nodes creates an ephemeral to announce there existence to the master. Now what i recognized is that after stopping the whole cluster and starting only the master again, still some ephemeral nodes may exists for some seconds. That leads me to following questions. What is if a node starts up again. Do it have to clean up it "old" ephemeral node, or can it somehow acquire the old one ? Just trying to find a best way how to deal with this, since on regular restart of the cluster i often recognize something like this master : node1, node2 connected master : node2 disconnected master : node2 connected again Thanks for any help Johannes ~~~ 101tec GmbH Halle (Saale), Saxony-Anhalt, Germany http://www.101tec.com