Jules Gosnell wrote:
Bruce Snyder wrote:
On 10/15/05, Jules Gosnell <[EMAIL PROTECTED]> wrote:
WADI currently sits on top of ActiveCluster, which it uses for
membership notification and ActiveMQ which is used for transport by
both layers. ActiveMQ has pluggable protocols, including a peer://
protocol which allows peers to talk directly to one another (this
should put to bed fears of a JMS based solution not scaling -
remember, JMS is just an API). So you do not need to choose between
WADI and ActiveCluster - they are complimentary. ActiveCluster can
also (I believe) use JGroups as a transport - I haven't tried it.
ActiveSpace is another technology in this area (distributed caching)
and it looks as if WADI and ActiveSpace will become more closely
aligned. So this may also be considered a complimentary technology.
Both Tomcat and Jetty currently have existing clustering solutions. I
looked closely at the Tomcat solutions before starting out on WADI
and knew all about the Jetty solution, because I wrote it :-). WADI
is my answer to what I see as shortcomings in all of the existing
open source approaches to this problem-space.
Can you provide a quick high level description of the advantages of
WADI over Tomcat and Jetty clustering solutions?
Jetty uses 1->all replication over jgroups, as I believe 1 Tomcat
session manager does. I think the other Tomcat session manager also
does
1->all replication, but over its own protocol. Perhaps Jeff can confirm
this. I think TC's 'PersistentManager' is also able to write changed
sessions out to disc at the end of the request.
1->all, for the reasons given above will not scale. The more nodes you
add, the more notifications each will have to react to and the more
sessions it will have to hold. you are simply deferring your problems
for a little while. Your only way out is to partition cluster and
sacrifice your availability. When WADI's in-vm replication strategy is
finished, I think that this will make it a clear winner for anyone
wishing to cluster more than 2-3 nodes.
I know that this may sound a bit off the wall, but what about
implementing the BitTorrent protocol
(http://www.bittorrent.com/protocol.html)? The purpose of the
BitTorrent protocol is to redistribute the cost of downloading a file
of (potentially) unlimited size to the downloaders themselves by
automatically recruiting the downloaders to upload chunks of the file
to their peer downloaders. There's a very good paper available
(http://www.bittorrent.com/bittorrentecon.pdf) that outlines the
BitTorrent protocol at a higher level and explains the economic theory
around which the protocol was built and how it has already been proven
through it's many implementations and high utilization throughout the
world today.
interesting idea :-)
Off the top of my head, I guess the main difference in usecase is the
scale of our requirement (generally much smaller objects over LAN,
instead of large objects over WAN) and the frequency with which they
change - all the time vs. never. Bittorent uses massive storage
redundancy to ensure the survival of cached items - I guess they can
do this because disc is cheap, whereas for an in-vm replication
scheme, we want to keep sessions in memory to keep things fast and the
possible footprint of each node is limited to a few gig - the smaller
the better. So we need to keep the number or replicas down to the bare
minimum.
That's not to say that we can't learn anything from bittorrent. I'll
give it some thought today and try to do some reading around it.
Cheers, Bruce
Jules
Of course, all this stuff will be abstracted out into pluggable
strategies anyway - so there is no reason why we shouldn't have
bittorrent-based replication strategy as well as whatever i have already
come up with :-)
Jules
Bruce
--
perl -e 'print
unpack("u30","D0G)[EMAIL PROTECTED]&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
The Castor Project
http://www.castor.org/
Apache Geronimo
http://geronimo.apache.org/
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
