Re: How to reestablish a session

[Gustavo Niemeyer]

 is_unrecoverable() means exactly that: the session is toast. nothing you do
 will get it back.

Ok, I was wondering about what exactly was unrecoverable indeed.

 zookeeper_init is almost never used with a non-null client_id. the main use
 case for it is crash recovery. i've rarely seen it used, but you can start a
 session, save off the client_id to disk, create ephemerals etc., then if
 your program crashes, you can restart and recover the session and pick back
 up where you left off. in this case we don't worry about the session being
 closed by the previous instance of the program because it crashed. it's
 pretty tricky to use.

Understood.  I agree this is a pretty unique case, and a very hard one
to get right by itself (how to get the app in the proper state to
receive watches after the whole application has crashed?).

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

RE: How to reestablish a session

[Fournier, Camille F. [Tech]]

This is exactly the scenario that you use to test session expiration, make one 
connection to a ZK and then another with the same session and password, and 
close the second connection, which causes the first to expire. It is only a 
clean close that will cause this to happen, though (one where the client calls 
close to end the connection). 

Right now, if you have a partition between client and server A, I would not 
expect server A to see a clean close from the client, but one of the various 
exceptions that cause the socket to close. These do not do anything currently 
to change the state of the session, and if the client connects elsewhere before 
the session timeout, the session will remain active.

C


-Original Message-
From: Gustavo Niemeyer [mailto:gust...@niemeyer.net] 
Sent: Thursday, November 18, 2010 10:16 AM
To: ZooKeeper Users
Subject: How to reestablish a session

Greetings,

As some of you already know, we've been using ZooKeeper at Canonical
for a project we've been pushing (Ensemble, http://j.mp/dql6Fu).
We've already written down txzookeeper (http://j.mp/d3Zx7z), to
integrate the Python bindings with Twisted, and we're also in the
process of creating a Go binding for the C ZooKeeper library (to be
released soon).

Yesterday, while working on the Go bindings, a test made me wonder
about what's the correct way to reestablish a session with ZooKeeper.

In another thread a couple of months ago, Ben mentioned:

 i'm a bit skeptical that this is going to work out properly. a server may
 receive a socket reset even though the client is still alive:

 1) client sends a request to a server
 2) client is partitioned from the server
 3) server starts trying to send response
 4) client reconnects to a different server
 5) partition heals
 6) server gets a reset from client

 at step 6 i don't think you want to delete the ephemeral nodes.

I also don't think it should delete ephemeral nodes.  While performing
some tests, though, I noticed that something similar to this may
happen.

The following sequence was performed in the test:

1) Establish connection A to ZK
2) Create an ephemeral node with A
3) Establish connection B to ZK, reusing the session from A
4) Close connection A
5) The ephemeral node from (2) got deleted.

So, this made me wonder about what's the proper way to reestablish a
session in practice, due to partitioning. Imagine that the
reconnection which happened on (3) was an attempt from the client to
restore the communication with the ZK cluster when faced with
partitioning.  Once the connection succeeded, the old resources from
connection A should be disposed, but how to do this without risking
killing the healthy connection on B (imagine that the network comes
back between (3) and (4)).

Anyone has thoughts on that?

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

Re: How to reestablish a session

[Gustavo Niemeyer]

 Right now, if you have a partition between client and server A, I would not 
 expect
 server A to see a clean close from the client, but one of the various 
 exceptions
 that cause the socket to close.

Please don't get me wrong, but I find it very funny to rely on the
stability of a network partition to avoid having a session killed.

Either way, that's not a big deal for me, now that I understand the
problem.  Knowing about it, I can simply postpone the close() until a
safe time.  It just felt worth pointing out, since this will arguably
be *very* hard to track down in practice.

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

Re: How to reestablish a session

[Benjamin Reed]

ah i see. you are manually reestablishing the connection to B using the 
session identifier for the session with A.


the problem is that when you call close on a session, it kills the 
session. we don't really have a way to close a handle without do that. 
(actually there is a test class that does it in java.)


if you want this, you should open a jira to do a close() without killing 
the session.


why don't you let the client library do the move for you?

ben


On 11/18/2010 11:51 AM, Gustavo Niemeyer wrote:

Hi Ben,


that quote is a bit out of context. it was with respect to a proposed
change.

My point was just that the reasoning why you believed it wasn't a good
approach to kill ephemerals in that old instance applies to the new
cases I'm pointing out.  I wasn't suggesting you agreed with my new
reasoning upfront.


in your scenario can you explain step 4)? what are you closing?

I'm closing the old ZooKeeper handler (zh), after a new one was
established with the same client id.