Hello Sanjaya,
comments are inline marked with [RA]
Rajith Attapattu
Red Hat.
On 2/5/07, Sanjaya Karunasena <[EMAIL PROTECTED]> wrote:
Hi All,
Since I am interested in distributed computing, I am closely monitoring
this
discussion on clustering. I am still going through the code to understand
the
approach/design/implementation. Since I am new to this codebase I have
very
little understanding about most of the implementations. However I hope
these
comments will help you all to define a scalable, flexible clustering
implementation for Axis2.
[RA] thank you for your interest and welcome to scene.
In a distributed scenario having the ability to handle concurrency is very
important. How do you tackle conflicting updates? There should be a way to
rollback/commit. In otherwords the interface should be transactionable.
Also
having the ability to distinguish between read-only vs read/write access
to
the context is an opportunity to apply some optimizations to the actual
implementation.
[RA] This is where locking comes into picture. There are 2 different types
of locking mechanisms, pesimistic locking and optimistic locking.
Most databases use pesimistic locking and is expensive and slow, while
optimistic locking is relatively cheap. Negotiating concurrency issues is
best handled by the underlying infrastructure. Apache Tribes (the impl I
used and is using total ordering) and Richochet (garuntees are lower than
VS) are both low level reliable multicasting infrastructure and have no
support for locking.
WADI from codehaus specializes in locking. But for now we decided sometime
back to go without any locking for the following reasons.
WADI is written for session replication and is difficult to adapt as is to
the context replication scenario we have with out some modifications.
Sanjiva and I debated on this issue for a while and agreed that we live with
the concurrency issues for the time being.
What we can do is to try to utilise WADI to provide locking support for our
implementation as the second step.
Detecting conflicts and rolling back is best decided at the infrastructure
level (for example WADI) rather than at the application level.
The logic for rolling back is complicated and if a rollback occurs we need
to notify the web service client. Currently there is no standard WS way of
doing that.
(Not sure WS-Context has anything regarding this)
We can have dirty reads (cheap) and do locking only for updates. But ymmv.
I suggest that this interface should provide some means to do
that too.
[RA] As I said, the application should not be figuring out wether to
rollback or not, the infrastructure should.
so it should be the reverse. there should be a way for the infrastructure to
notify the client about a rollback.
I think the intention of abstracting the cluster manager to an interface is
to
provide extensibility so that we can have different cluster manager
implementations based on different group communication techniques. But I
think right now the interface has introduced more coupling.
Following is an approach which could reduce coupling and enable more
flexibility and extensibility.
Do we have the ability to register a listener to the Context? What I mean
is
an instance of a class which implements an interface ContextListner. The
ContextListner has one method "notify" which take a parameter of
type "Event". The ClusterManager interface can extend the ContextListner
(BTW, Should ClusterManger be an abstract class?)
[RA} unfortunately your replication points can be outside of the context
heirachy.
For example instead of replicating every add/remove change happening inside
an invocation we can choose to replicate the net result (change in state) at
the end of the invocation. I think this is more efficient. The decesion to
replicate something should not be limited to be taken within the context
heirachy only.
There should be flexibility to call the ClusterManager from anywhere. So a
context listener is not sufficient.
Furthermore your coupling is only reduced to the cluster manager. I also
didn't like the prospect of calling cluster manager from all over the place,
but given the current design of the context heirachy we have no choice.
The other more cleaner approach that I proposed a while back was to use
Aspects. We can use aspects to capture replication points using load time
weaving in a more non intrusive manner. This way the context code is not
littered with calls to the Cluster manager. I do have experimental version
on my local copy, but it still needs more work.
This way, in a particular node when there is change in the Context the
ClusterManager instance in that node get notify. Then that ClusterManger
instance can mulitcast (reliable mulitcasting) the change to other
ClusterManger instances running in other nodes. Since each ClusterManager
instance has a handler to the Context it can apply the change locally. If
the
reliable multicast is configured to keep the "total order" in message
ordering you don't have to worry about the conflicts due to concurrency.
[RA] This is what happens currently. Context creations are notified
immediatly and state updates are notified at the end of an invocation.
Tribes which is a total odering reliable multicast then does the work.
However this will not prevent concurrency issues completely. Unless we do
pesimistic locking we would still experiance dirty reads and potential
write/write conflicts.
This approach can be further optimized/enhanced based on different usecases.
Thanks and best regards,
Sanjaya
* There is no difference
On Monday 05 February 2007 10:35, Rajith Attapattu wrote:
> Chamikara,
>
> The code is available under branches/clustering.
>
> Here are the changes I made to ClusterManager interface. Of course what
I
> did is just a prototype so feel free to experiment and change things.
>
> public abstract void init (ConfigurationContext context);
> public abstract void addContext (AbstractContext context);
> public abstract void removeContext (AbstractContext context);
>
> public abstract void updateState (AbstractContext context);
>
> As you can the changes are where I replaced the add/remove/touch
property
> methods with a single updateState method.
> I found it to be very cumbersome/expensive to intercept the property
> changes and keep track of them on a per occurence basis.
> Instead a single updateState method called at the end of the invocation
> provides better alternative to handle the state changes. YMMV.
>
> looking forward to your comments on the prototype.
>
> Regards,
>
> Rajith
>
> On 2/2/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> > Yes the implementation is build on top of the cluster manager.
> > I changed the method signatures slightly. But it's only a prototype.
> >
> > I will not check it into the trunk, untill we run a successfull demo
and
>
> everybody is comfortable.
>
> > I will put that in a branch and then you guys can also make changes.
> >
> > Regards,
> >
> > Rajith
> >
> > On 2/2/07, Chamikara Jayalath < [EMAIL PROTECTED]> wrote:
> > > Rajith,
> > >
> > > What we are adding here is the interface that we agreed sometime
back.
>
> So nothing prevents your implementation from working on top of that.
>
> > > BTW, what were the modifications that were done to the interface.
> > >
> > > Chamikara
> > >
> > > On 2/2/07, Rajith Attapattu < [EMAIL PROTECTED]> wrote:
> > > > Chamikara,
> > > >
> > > > I do have half baked impl sitting on my laptop based on a slightly
>
> modified cluster manager.
>
> > > > I will soon check this into a branch and you guys can then start
>
> playing around with it.
>
> > > > Sorry about this taking time, I was so occupied with the Qpid
work.
> > > >
> > > > Regarda,
> > > >
> > > > Rajith
> > > >
> > > > On 2/1/07, Chamikara Jayalath < [EMAIL PROTECTED]> wrote:
> > > > > Hi All,
> > > > >
> > > > > As we discussed and agreed sometime back we will be adding an
>
> abstraction layer to Axis2 to make it possible to transparently plug-in
> different clustering implementations. Our abstraction was named
> 'ClusterManager' and the interface that was proposed is given in [1]
(with
> some slight modifications).
>
> > > > > I am hoping to go a step forward my adding this interface to the
>
> Axis2 code base. At initiation Axis2 will load a ClusterManager
> implementation object (if configured) and will call the methods of it
> whenever respective events occur.
>
> > > > > Chamikara
> > > > >
> > > > >
> > > > > [1] ClusterManager interface
> > > > >
> > > > > public abstract class ClusterManager {
> > > > >
> > > > > public abstract void init (ConfigurationContext context);
> > > > > public abstract void addContext (AbstractContext context);
> > > > > public abstract void removeContext (AbstractContext
context);
> > > > > public abstract void addProperty (AbstractContext context,
>
> String propertyName, Object propertyValue);
>
> > > > > public abstract void removeProperty (AbstractContext
context,
>
> String propertyName);
>
> > > > > public abstract void touchProperty (AbstractContext context,
>
> String propertyName);
>
> > > > > public abstract void commit ();
> > > > >
> > > > > }
--
Senior Software Architect
WSO2 Inc.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
