Filip,
This is good feedback; I wasn't aware of the details on what Tomcat
has done in this area.
We added concept of location to the Session APIs to address a few of
the weird and surprisingly common use cases.
1) Unreliable load balancer
In this case, the load balancer is not 100% and sometimes delivers
requests to the wrong node.
The solution to the first part is to allow the container to discover
the actual location of the session and do a client or server side
redirect to the correct node.
2) False positive on server failure
Load balancer decides that the machine has died and picks a new
random node for the request.
This requires either moving the session to the local machine or
sending a message to the load balancer informing it of the correct
location for the request.
3) Geronimo based load balancer
A Geronimo node sits in front of a group of Geronimo server, receives
all requests and directs them to the correct node.
In this case, the server needs to know the location of the session
and proxies the request to the correct node.
4) Smart Balancing
Some force behind the scenes (under the covers of the APIs) is
manipulating the load directly by migrating sessions. The clients
may not be aware of the migration and send requests to the wrong node.
This is very similar to Case 2, except in this case we need a way to
inform the client of the correct location of the session. The
pending request can simply be proxied on to the correct node with a
side-band message that the next request should go to a different
node, or we could do a client side redirect.
As you can, see all of the use cases we thought of require the
knowledge of the session location. Now, I am willing to admit that
this may be a "golden hammer" solution, so I'm open to any other ways
to implement these features without having to know session location.
I have some comments inline about the session api you present.
-dain
On Mar 3, 2006, at 7:00 AM, Filip Hanik - Dev Lists wrote:
gentlemen, not a committer here, but wanted to share some thoughts.
in my opinion, the Session API should not have to know about
clustering or session replication, nor should it need to worry
about location.
the clustering API should take care of all of that.
the solution that we plan to implement for Tomcat is fairly
straight forward. Let me see if I can give an idea of how the API
shouldn't need to worry, its a little lengthy, but it shows that
the Session and the SessionManager need to know zero about
clustering or session locations. (this is only one solution, and
other solutions should demonstrate the same point, SessionAPI need
to know nothing about clustering or session locations)
1. Requirements to be implemented by the Session.java API
bool isDirty - (has the session changed in this request)
bool isDiffable - is the session able provide a diff
byte[] getSessionData() - returns the whole session
byte[] getSessionDiff() - optional, see isDiffable, resets the
diff data
void setSessionDiff(byte[] diff) - optional, see isDiffable,
apply changes from another node
To throw you arguments back on you, why should my code be exposed to
this level of detail :) From my perspective, I get a session and it
is the Session API implementation's problem to figure out how to diff
it, back it up, and migrate it.
2. Requirements to be implemented by the SessionManager.java API
void setSessionMap(HashMap map) - makes the map implementation
pluggable
3. And the key to this, is that we will have an implementation of a
LazyReplicatedHashMap
The key object in this map is the session Id.
The map entry object is an object that looks like this
ReplicatedEntry {
string id;//sessionid
bool isPrimary; //does this node hold the data
bool isBackup; //does this node hold backup data
Session session; //not null values for primary and backup nodes
Member primary; //information about the primary node
Member backup; //information about the backup node
}
The LazyReplicatedHashMap overrides get(key) and put(id,session)
Why would anyone need to know this level of detail?
So all the nodes will have the a sessionId,ReplicatedEntry
combinations in their session map. But only two nodes will have the
actual data.
This solution is for sticky LB only, but when failover happens, the
LB can pick any node as each node knows where to get the data.
The newly selected node, will keep the backup node or select a new
one, and do a publish to the entire cluster of the locations.
I don't see anyway to deal with locking or the fact that servlet
sessions are multi threaded (overlaping requests). How do you know
when the session is not being used by anyone so you have a stable
state for replication.
As you can see, all-to-all communications only happens when a
Session is (created|destroyed|failover). Other than that it is
primary-to-backup communication only, and this can be in terms of
diffs or entire sessions using the isDirty or getDiff. This is
triggered either by an interceptor at the end of each request or by
a batch process for less network jitter but less accuracy (but
adequate) for fail over.
As you can see, access time is not relevant here, nor does the
Session API even know about clustering.
How do you deal with access-time? I agree that your API doesn't know
about clustering, but you also can't do a client side or server side
redirect to the correct node; you must always migrate the session to
your request.
In tomcat we have separated out group communication into a separate
module, we are implementing the LazyReplicatedHashMap right now
just for this purpose.
Cool. I'm interested to see what you come up with.
-dain
