Re: How to reestablish a session
is_unrecoverable() means exactly that: the session is toast. nothing you do will get it back. Ok, I was wondering about what exactly was unrecoverable indeed. zookeeper_init is almost never used with a non-null client_id. the main use case for it is crash recovery. i've rarely seen it used, but you can start a session, save off the client_id to disk, create ephemerals etc., then if your program crashes, you can restart and recover the session and pick back up where you left off. in this case we don't worry about the session being closed by the previous instance of the program because it crashed. it's pretty tricky to use. Understood. I agree this is a pretty unique case, and a very hard one to get right by itself (how to get the app in the proper state to receive watches after the whole application has crashed?). -- Gustavo Niemeyer http://niemeyer.net http://niemeyer.net/blog http://niemeyer.net/twitter
How to reestablish a session
Greetings, As some of you already know, we've been using ZooKeeper at Canonical for a project we've been pushing (Ensemble, http://j.mp/dql6Fu). We've already written down txzookeeper (http://j.mp/d3Zx7z), to integrate the Python bindings with Twisted, and we're also in the process of creating a Go binding for the C ZooKeeper library (to be released soon). Yesterday, while working on the Go bindings, a test made me wonder about what's the correct way to reestablish a session with ZooKeeper. In another thread a couple of months ago, Ben mentioned: i'm a bit skeptical that this is going to work out properly. a server may receive a socket reset even though the client is still alive: 1) client sends a request to a server 2) client is partitioned from the server 3) server starts trying to send response 4) client reconnects to a different server 5) partition heals 6) server gets a reset from client at step 6 i don't think you want to delete the ephemeral nodes. I also don't think it should delete ephemeral nodes. While performing some tests, though, I noticed that something similar to this may happen. The following sequence was performed in the test: 1) Establish connection A to ZK 2) Create an ephemeral node with A 3) Establish connection B to ZK, reusing the session from A 4) Close connection A 5) The ephemeral node from (2) got deleted. So, this made me wonder about what's the proper way to reestablish a session in practice, due to partitioning. Imagine that the reconnection which happened on (3) was an attempt from the client to restore the communication with the ZK cluster when faced with partitioning. Once the connection succeeded, the old resources from connection A should be disposed, but how to do this without risking killing the healthy connection on B (imagine that the network comes back between (3) and (4)). Anyone has thoughts on that? -- Gustavo Niemeyer http://niemeyer.net http://niemeyer.net/blog http://niemeyer.net/twitter
RE: How to reestablish a session
This is exactly the scenario that you use to test session expiration, make one connection to a ZK and then another with the same session and password, and close the second connection, which causes the first to expire. It is only a clean close that will cause this to happen, though (one where the client calls close to end the connection). Right now, if you have a partition between client and server A, I would not expect server A to see a clean close from the client, but one of the various exceptions that cause the socket to close. These do not do anything currently to change the state of the session, and if the client connects elsewhere before the session timeout, the session will remain active. C -Original Message- From: Gustavo Niemeyer [mailto:gust...@niemeyer.net] Sent: Thursday, November 18, 2010 10:16 AM To: ZooKeeper Users Subject: How to reestablish a session Greetings, As some of you already know, we've been using ZooKeeper at Canonical for a project we've been pushing (Ensemble, http://j.mp/dql6Fu). We've already written down txzookeeper (http://j.mp/d3Zx7z), to integrate the Python bindings with Twisted, and we're also in the process of creating a Go binding for the C ZooKeeper library (to be released soon). Yesterday, while working on the Go bindings, a test made me wonder about what's the correct way to reestablish a session with ZooKeeper. In another thread a couple of months ago, Ben mentioned: i'm a bit skeptical that this is going to work out properly. a server may receive a socket reset even though the client is still alive: 1) client sends a request to a server 2) client is partitioned from the server 3) server starts trying to send response 4) client reconnects to a different server 5) partition heals 6) server gets a reset from client at step 6 i don't think you want to delete the ephemeral nodes. I also don't think it should delete ephemeral nodes. While performing some tests, though, I noticed that something similar to this may happen. The following sequence was performed in the test: 1) Establish connection A to ZK 2) Create an ephemeral node with A 3) Establish connection B to ZK, reusing the session from A 4) Close connection A 5) The ephemeral node from (2) got deleted. So, this made me wonder about what's the proper way to reestablish a session in practice, due to partitioning. Imagine that the reconnection which happened on (3) was an attempt from the client to restore the communication with the ZK cluster when faced with partitioning. Once the connection succeeded, the old resources from connection A should be disposed, but how to do this without risking killing the healthy connection on B (imagine that the network comes back between (3) and (4)). Anyone has thoughts on that? -- Gustavo Niemeyer http://niemeyer.net http://niemeyer.net/blog http://niemeyer.net/twitter
Re: How to reestablish a session
Right now, if you have a partition between client and server A, I would not expect server A to see a clean close from the client, but one of the various exceptions that cause the socket to close. Please don't get me wrong, but I find it very funny to rely on the stability of a network partition to avoid having a session killed. Either way, that's not a big deal for me, now that I understand the problem. Knowing about it, I can simply postpone the close() until a safe time. It just felt worth pointing out, since this will arguably be *very* hard to track down in practice. -- Gustavo Niemeyer http://niemeyer.net http://niemeyer.net/blog http://niemeyer.net/twitter
Re: How to reestablish a session
ah i see. you are manually reestablishing the connection to B using the session identifier for the session with A. the problem is that when you call close on a session, it kills the session. we don't really have a way to close a handle without do that. (actually there is a test class that does it in java.) if you want this, you should open a jira to do a close() without killing the session. why don't you let the client library do the move for you? ben On 11/18/2010 11:51 AM, Gustavo Niemeyer wrote: Hi Ben, that quote is a bit out of context. it was with respect to a proposed change. My point was just that the reasoning why you believed it wasn't a good approach to kill ephemerals in that old instance applies to the new cases I'm pointing out. I wasn't suggesting you agreed with my new reasoning upfront. in your scenario can you explain step 4)? what are you closing? I'm closing the old ZooKeeper handler (zh), after a new one was established with the same client id.