FYI - A Thrift proxy for Curator will be in Curator’s next release. I mentioned
it on this list a few weeks ago:
http://mail-archives.apache.org/mod_mbox/zookeeper-user/201406.mbox/%[email protected]%3E
-Jordan
From: Van Klaveren, Brian N. [email protected]
Reply: [email protected] [email protected]
Date: June 26, 2014 at 3:03:31 PM
To: [email protected] [email protected]
Subject: An updated ZooKeeper RESTful API - Zest
Hi All,
I’m new to this list.
I was wondering who I could contact, if this list isn’t appropriate, to
possibly discuss design decisions of an update to the RESTful API.
I recently started working on an update and partial rewrite of the ZooKeeper
RESTful API, using Jersey 2 namely, and with support for asynchronous
requests/long polling on events. I did this after exploring using etcd
possibly, but realizing their API is rather inadequate for locking, and overall
the project isn’t very mature. In order to prevent some confusion, I’ve named
my project “Zest”, though it largely derives from the current RESTful API.
One major difference with Zest is that there is an “events” resource. This is a
resource a client can poll or long-poll to receive events. My hope was that
Zest, through the events resource, could initially support polling and
long-polling, with the possibility of plugging in WebSockets (and maybe
Server-Side events) in the future. I actually played around with support for
Server-Side events, but I was unable to easily program a responsive client in
JavaScript or Python.
I did look into integrating some of this into the existing ZooKeeper server
somehow in order to reduce the additional memory and processing overhead, but
that didn’t appear to me to be easy or straightforward at the level of
abstraction that currently exists.
On the other hand, I have created a PoC multi-threaded client based on zkclient
which appears to work similarly, except that it interacts only with Zest.
If this all sounds like it could be useful, I’d be interested in collaborating
with people to work on the project and discussing an API.
Brian
Some info on differences in the Zest implementation:
A Zest session is effectively an HTTP session currently. This could be changed,
but I just used this for ease of implementation. Some people might say this
isn’t “RESTful”, but I think that’s okay.
A Zest session is fully initiated when a client POSTs to the sessions resource.
This creates a ZkSessionProxy object, which mainly handles the ZooKeeper
client, schedules, cancels and resets ZooKeeper client reapers, and temporarily
stores suspended AsyncResponses for long polling, as well as watched events for
the events resource. In the future, it could possibly maintain dispatching
events and via WebSockets.
The Zest implementation currently defines it’s own ticks, where the tickTime is
half of the ZooKeeper session timeout. A ZooKeeper client reaper is scheduled
to be executed at two ticks, or effectively the same time as the session
timeout.
Every incoming HTTP request cancels and reschedules the ZooKeeper client
reaper, similar to the current way. A GET request on the sessions resource will
act as a generic keep-alive.
With long-polling, there is a asynchronous request timeout set on a one-tick
interval. When that timeout occurs, a “204 NO CONTENT” is sent, and the client
should renew it’s request. When that request is renewed, the reaper is canceled
and then reset. In this sense, a thread that only watches for events functions
as a heartbeat.
If a ZooKeeper client session is reaped, then the ZooKeeper client is closed.
Then, a Zest session reaper is scheduled. This is to allow clients which are
unresponsive to discover that they have, in fact, been disconnected when
polling the events resource, as well as retrieve any events which may have
arrived to the HTTP server before they were disconnected, if they are able to
query the web service within a few ticks. Once the Zest session reaper is run,
it removes all events and references to the Zest session
