Rajith Attapattu wrote:
More question if you don't mind.
> 2.) Assuming sombody wants to do session replication (All
> Active) instead of (one Active and "n" backups) is there provision
> within the WADI api to plug in this stratergy?
>I'm giving this some thought in terms of SFSB support, I'm not aware of
>similar constraints in the EJB world...
>I guess we could relax this constraint in the web world, but I am not
>sure that I think that this is a good idea. Can you see a way to do this
>and maintain spec compliance and performance ?
Is WADI designed primarily for Web?? (bcos u talked about being
servlet spec compliant) and u also mention about SFSB support.
WADI was initially designed for the web - because I saw the issues
surrounding HttpSession distribution, particularly the requirement for a
single 'active' session as unresolved in any open source offering and I
thought it was about time that there was a truly compliant solution.
It soon became clear that many of the problems faced by sessions in the
web-tier were also faced by sessions in the EJB-tier...
Can we abstract the Replication problem to a more higher level and
have the two (or more if there is) stratergies as impls of the
replication API that installs as a pluggin by the user.
Well, we could, but you would have to convince me that SFSBs would
benefit from a 'multiple-active-sessions' approach... I haven't given it
much thought, but I don't see any advantage - bear with me :
- in the EJB world, we own the client side proxy. We can impose strict
affinity. An invocation arriving at a node that is not carrying the
primary session copy will be an exceptional occurance.
- If you go with the 'single-active-session' model, and an invocation
does land on a secondary, you then pay an exceptional cost -
deserialisation and promotion (messaging) from secondary to primary.
This is OK, since you are in an exceptional situation.
- If you go with the 'multiple-active-sessions' approach you have two
choices regarding deserialisation of replication messages.
1) you can deserialise them as they arrive - bad idea, because
deserialisation is extremely expensive and most of the time these copies
will never be used.
2) you can deserialise them lazily - only bother to do the work, if/when
an invocation arises.
Regardless of which you choose (and I hope you would choose 2), you are
now in a situation where two copies of a session may diverge from each
other. Lets say you make a change to one, then you make a change to the
other, but the replication message from the first session arrives at the
second session after your second change and wipes it out, and the
replication message from your second change then overwrites the first
session with what is now a different value than that carried by the
second.... you can detect these issues by versioning, but the best way
to protect against them occuring (see reasons for needing a pessimistic
algorithm below) is by having some form of distributed locking. In
effect, the guy with the lock is the primary and the guy without it the
secondary.
OK, so now we have a working 'multiple-active-sessions' model - but hold
on, it is doing lazy deserialisation and distributed locking - it looks
very like the 'single-active-session' model....
Does that help ?
We can abstract things like a ReplicationManager that handles/controls
no of replicas etc.. and a ReplicatedSession which decides wether it's
active or passive (backup) based on the parameters passed to the
ReplicatedSessionFactory at create time from the ReplicationManager.
sure - and all of these things are already pluggable in WADI.
The ReplicationManager impl could be the stratergy that decides wether
it maintains n of active replicas or 1 active and n backups or any
other stratergy.
Yes it could - but I think that this is still being driven by your
attachment to the multiple-active-sessions model and I do not see that
as viable.
Also the ReplicatedSession could impl stratergies like in
MemoryReplication or PassiveReplication (based on active or passive)
or anything else. And PassiveReplication can be extended to file
based, database backed (not recomended) or anything else.
WADI's replication strategy is already pluggable. We have a basic DB
replication scheme and are working on the in-vm scheme. Other schemes
could easily be added.
If we open up the API and let the user choose the stratergy they want
then we are delaying our concerns to the user level and allow them to
make the decesion.
I am sure we cannot address every situation, and the user is the best
to judge about there env.
But we can always provide some sensible stratergies and recomendations
and use cases around them to make an informed decesion.
Then We can leave the decesion to the user about
spec-complient/performance.
What do u think??
Unless you can demonstrate a clear win for a strategy that is
non-compliant, I would be very hesitant to ship one.
WADI is designed so that pretty much everything that you might want to
plug is pluggable. But the larger the piece that you want to plug in,
the more work you would have to do writing it and making sure that it
did not collide with any other fn-ality.
>If a request arrives at a secondary, primary and secondary swap roles
>and processing happens locally.
>If a request arrives on a node with no copy of the relevant session, it
>may be relocated to the primary, or the primary to it.
1. Do u plan to have an abstraction around the above concerns as well??
So we can have impls of different stratergies, So people can
decide wether they want to relocate the primary or the request.
yes - this decision is pluggable.
In case of a relocation of either request or session I assume u
have hidden the impls behind an interface/API sort of thing so ppl can
do different impls of the same stratergies or impl their own stratergy.
yes
2. In the event of a primary and secondary swapping roles or having n
of active replicas don't we need some sort of distributed locking
mechanism.
I heard that in memory locking should be optimistic and storage backed
replicas should be pessimistic locking.
session locking has to be pessimistic, because changes are made by app,
not container code. So a collision of changes could not be resolved by
the container, so it cannot be allowed to happen.
WADI contains a distributed locking mechanism within its Partitioning
system. When a copy is promoted, a message will pass from it (containing
its version number), to its partition (where it will take a lock and
find the location of the primary), on to the primary (where it will
compare version numbers), back to the secondary (with a possible update,
if its version is out of date) and finally back to the partition (where
the primary's new location will be stored and the lock released). local
locking will also occur around both secondary and primary whilst they
are involved in this interaction.
I hope I haven't got the too mixed up :)
No, I don' think so, but I do think that you need to take a careful look
at exactly how you think a mutiple-active-sessions model might work and
whether this would, in fact, be any different from the model that I am
proposing.
Can u please touch on this problem as my knoweldge is limited on this
area.
is this enough detail ? :-)
Jules
Regards,
Rajith.
On 1/17/06, *Jules Gosnell* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
Rajith Attapattu wrote:
>
> Hi,
>
> Some of these questions came up after reading the thread on totem.
> However I started the new thread so that searching is easy and also
> want distract the intense discussions on totem with out-of-topic
> questions.
>
> Jules Gosnel wrote
>
> >This is not something that is really considered a significant
saving in
> >WADI (see my last posting's explanation of why you only want one
> >'active' copy of a session). WADI will keep session backups
serialised,
> >to save resources being constantly expended deserialising session
> >backups that may never be accessed. I guess actually, you could
consider
> >that WADI will do a lazy deserialisation in the case that you have
> >outlined, as primary and secondary copies will actually swap
roles with
> >attendant serialisation/passivation and deserialisation/activation
> >coordinated by messages.
>
> >If you are running a reasonable sized cluster ( e.g. 30 nodes -
it's all
> >relative) with a small number of backups configured ( e.g. 1),
then, in
> >the case of a session affinity brekdown (due to the leaving of a
> >primary's node), you have a 1/30 chance that the request will
hit the
> >primary, a 1/30 that you will hit the secondary and a 28/30
that you
> >will miss :-) So, you are right :-)
>
> So just to figure out if I understand this correctly.
>
> 1.) WADI only has one active and one-two backups at most (I
assume the
> no of backups is configurable)
replication is under implementation at the moment. Any number of
backups
should be configurable, but the more you have the less performant you
are. You trade off safety for speed.
>
> 2.) WADI is built up on the assumption of session affinity. So the
> probability of missing the primary and the secondary backup(s) goes
> up as the cluster grows according to your example
WADI will work without session affinity, however, as you would expect,
this will not perform as well as it might. If you switch on affinity,
you will drastically cut down the amount of request/session relocation
and most interactions should become local.
Switch off affinity, and of course, your chances of hitting a copy of
the session will go down. There are a fixed number of sessions and
you
are increasing the number of nodes... If you are intending to use
an lb
without affinity, then you should really reconsider. The costs are
tiny
and the gains enormous. Affinity is a standard feature on any serious
HTTP LB.
>
> 3.) How does WADI handle a situation where there is no session
affinity??
If a request lands on the primary, processing occurs locally.
If a request arrives at a secondary, primary and secondary swap roles
and processing happens locally.
If a request arrives on a node with no copy of the relevant
session, it
may be relocated to the primary, or the primary to it.
>
> 4.) Have you compared the overhead of maintaining session
affinity vs
> having R replicas (all-Active) to service the client.
I have worked on impls using both approaches and am satisfied that my
most recent approach will be the most performant.
>
> >If, however, you did your deserialisation of replicants up
front and
> thus avoided further messages when a secondary was hit, by
maintaining
> >all copies 'active' (I think you would not be spec compliant if you
> did this),
>
> 1.) What do u mean by spec here ?? Are u talking about the WADI
spec?
There is no WADI spec :-) - I'm talking about the servlet spec -
specifically :
SRV 7.7.2 - "Within an application marked as distributable, all
requests
that are part of a session must be handled by one Java Virtual
Machine1
( JVM ) at a time." and "Containers must notify any session attributes
implementing the HttpSessionActivationListener during migration of a
session. They must notify listeners of passivation prior to
serialization of a session, and of activation after
deserialization of a
session."
These two constraints make it, IMHO, much more difficult to try
implementing any system that maintains multiple 'active', or
'primary'
copies of a session. The system needs to be absolutely clear where the
single 'active' copy is at any one time, in order to remain compliant.
To ensure that activation/passivation semantics work OK, only this
session may be activated, whilst the other 'secondary' copies are
passivated. By leaving the secondaries in serialised form, you save
further cycles and arrive at WADI's current design.
>
> 2.) Assuming sombody wants to do session replication (All
> Active) instead of (one Active and "n" backups) is there provision
> within the WADI api to plug in this stratergy?
I'm giving this some thought in terms of SFSB support, I'm not
aware of
similar constraints in the EJB world...
I guess we could relax this constraint in the web world, but I am not
sure that I think that this is a good idea. Can you see a way to
do this
and maintain spec compliance and performance ?
>
> If u remeber we talked about extention points within WADI.
>
> 1.) Is there a doc that describes WADI architecture
Not as yet, just a website with various resources hanging of it.
WADI is
still relatively young. The best source of architecture info is the
conversations that we have been having.
>
> 2.) Is there a doc that describes these extention points and how
to do
> it?? (Looking for a little more info than the API doc)
WADI is put together using Spring. You just check out the javadoc and
plug the pojos together. A lot of what we have been talking about is
architectural design and not implemented (although the
primary/secondary
stuff is all in and working).
regards,
Jules
>
> Thanks,
>
> Rajith.
>
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net <http://www.coredevelopers.net>
*
* Open Source Training & Support.
**********************************/
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
