Re: [AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)

2006-07-03 Thread Nathan Folkman
Speaking of SOB, we're working on getting the code to our dci (Digital 
City) module ready for release to the Open source community. This 
module contains the following items which folks may find interesting:


* SOB - Dossy described this below. Basically it's what we use at AOL to 
forward cache content.

* Network Poll - Provides a simple mechanism to create member facing polls.
* Network Tally - Allows one to aggregate counts for keys. We've used 
this in the past to track things like user views of a particular venue.
* Network Comment Boards - Allows one to easily create and add comment 
boards to your application.
* Network NV (network variables) - Similar to nsv, but allows updates to 
be distributed to many clients. Allows you to update variables on a 
single NV writer, and have those changes broadcast out to many front-end 
readers.
* Network Eval - Allows you to send Tcl scripts to be evaluated to many 
other servers.


Those are the really big things. They've been at use at AOL since the 
early Digital City days, and although relatively simplistic in their 
design - most assume single-writer for instance, they have shown 
themselves to be quite robust.


Stay tuned for the release of the nsdci module! ;-)

- n


[EMAIL PROTECTED] wrote:

On 2006.07.03, John Buckman [EMAIL PROTECTED] wrote:
  

In my experience, many  applications that use SQL actually only need
key-lookup capability [...]



Which is why, at AOL, a large part of the web publishing architecture
utilizes Small Object Broker (SOB), which can be set up in a distributed
(master-slave style) network based service with client-side read
caching with flush-on-write.  This lets you simply scale your key-based
query infrastructure horizontally.

Initially, I think some of the conventional wisdom at the
high-scalability end runs counter to what the majority of web developers
have discovered and learned because almost any reasonable solution will
work in the small which is where most web projects live.  In those
problem sets, tools and solutions which have a shallower learning curve
and offer more functionality out of the box yield faster development
times and empower developers to be productive with a smaller skill set.
However, there's a line (I couldn't yet tell you exactly where it is),
that once you cross it, the characteristics of the problem set changes
and the optimal (and sometimes only) solutions become very specialized.
It's at this end of the spectrum where AOLserver truly shines, but it's
a very small set.

John, it's fantastic to see you're actively looking at and working with
AOLserver.  From the list archives, it looks like your first message
came through around 15 Apr 2006.  I'm hoping that your interest in
AOLserver might mean that the upcoming version of Lyris may replace
tclhttpd with AOLserver?  Or are you working with AOLserver for
something different (i.e., Magnatune)?

-- Dossy

  



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] 
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: 
field of your email blank.


Re: [AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)

2006-07-03 Thread Tom Jackson
On Monday 03 July 2006 07:56, Nathan Folkman wrote:

 * Network NV (network variables) - Similar to nsv, but allows updates to
 be distributed to many clients. Allows you to update variables on a
 single NV writer, and have those changes broadcast out to many front-end
 readers.
 * Network Eval - Allows you to send Tcl scripts to be evaluated to many
 other servers.

Yumm! 

NVs and Network Eval sound ideal for pushing dynamic content into semi-static 
structures (cache?) The simple example is time-of-day, which updates maybe 
once per minute, so if you have hundreds or thousands or hundreds of 
thousands of requests per minute, this dynamic data is really static for one 
minute. A city's weather forcast doesn't change quite as often.

As far as scalability goes, the discussion has focused on components. However, 
I would like to mention that on the high end, this really ends up being about 
architecture and protocol, or how can the components be separated and how can 
they communicate. If the components themselves do not have built in 
communication APIs, then they becomes inseparable from other components.

It sounds like the nsdci module would extend the architecture of AOLserver 
significantly for anyone interested in scalablility. Maybe overall, the term 
scalability is something which applies to an application, not the components 
which are used to create the application. Components themselves might have an 
overall efficiency which measures how much time it takes to perform a given 
task. Eventually a given processor/thread/server will run out of resources or 
time. Scalability might be a measure of how easy it is to scale beyond a 
single instance of a component.

tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] 
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: 
field of your email blank.