[AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)
On 2006.07.03, John Buckman [EMAIL PROTECTED] wrote: In my experience, many applications that use SQL actually only need key-lookup capability [...] Which is why, at AOL, a large part of the web publishing architecture utilizes Small Object Broker (SOB), which can be set up in a distributed (master-slave style) network based service with client-side read caching with flush-on-write. This lets you simply scale your key-based query infrastructure horizontally. Initially, I think some of the conventional wisdom at the high-scalability end runs counter to what the majority of web developers have discovered and learned because almost any reasonable solution will work in the small which is where most web projects live. In those problem sets, tools and solutions which have a shallower learning curve and offer more functionality out of the box yield faster development times and empower developers to be productive with a smaller skill set. However, there's a line (I couldn't yet tell you exactly where it is), that once you cross it, the characteristics of the problem set changes and the optimal (and sometimes only) solutions become very specialized. It's at this end of the spectrum where AOLserver truly shines, but it's a very small set. John, it's fantastic to see you're actively looking at and working with AOLserver. From the list archives, it looks like your first message came through around 15 Apr 2006. I'm hoping that your interest in AOLserver might mean that the upcoming version of Lyris may replace tclhttpd with AOLserver? Or are you working with AOLserver for something different (i.e., Magnatune)? -- Dossy -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)
Speaking of SOB, we're working on getting the code to our dci (Digital City) module ready for release to the Open source community. This module contains the following items which folks may find interesting: * SOB - Dossy described this below. Basically it's what we use at AOL to forward cache content. * Network Poll - Provides a simple mechanism to create member facing polls. * Network Tally - Allows one to aggregate counts for keys. We've used this in the past to track things like user views of a particular venue. * Network Comment Boards - Allows one to easily create and add comment boards to your application. * Network NV (network variables) - Similar to nsv, but allows updates to be distributed to many clients. Allows you to update variables on a single NV writer, and have those changes broadcast out to many front-end readers. * Network Eval - Allows you to send Tcl scripts to be evaluated to many other servers. Those are the really big things. They've been at use at AOL since the early Digital City days, and although relatively simplistic in their design - most assume single-writer for instance, they have shown themselves to be quite robust. Stay tuned for the release of the nsdci module! ;-) - n [EMAIL PROTECTED] wrote: On 2006.07.03, John Buckman [EMAIL PROTECTED] wrote: In my experience, many applications that use SQL actually only need key-lookup capability [...] Which is why, at AOL, a large part of the web publishing architecture utilizes Small Object Broker (SOB), which can be set up in a distributed (master-slave style) network based service with client-side read caching with flush-on-write. This lets you simply scale your key-based query infrastructure horizontally. Initially, I think some of the conventional wisdom at the high-scalability end runs counter to what the majority of web developers have discovered and learned because almost any reasonable solution will work in the small which is where most web projects live. In those problem sets, tools and solutions which have a shallower learning curve and offer more functionality out of the box yield faster development times and empower developers to be productive with a smaller skill set. However, there's a line (I couldn't yet tell you exactly where it is), that once you cross it, the characteristics of the problem set changes and the optimal (and sometimes only) solutions become very specialized. It's at this end of the spectrum where AOLserver truly shines, but it's a very small set. John, it's fantastic to see you're actively looking at and working with AOLserver. From the list archives, it looks like your first message came through around 15 Apr 2006. I'm hoping that your interest in AOLserver might mean that the upcoming version of Lyris may replace tclhttpd with AOLserver? Or are you working with AOLserver for something different (i.e., Magnatune)? -- Dossy -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)
On Monday 03 July 2006 07:56, Nathan Folkman wrote: * Network NV (network variables) - Similar to nsv, but allows updates to be distributed to many clients. Allows you to update variables on a single NV writer, and have those changes broadcast out to many front-end readers. * Network Eval - Allows you to send Tcl scripts to be evaluated to many other servers. Yumm! NVs and Network Eval sound ideal for pushing dynamic content into semi-static structures (cache?) The simple example is time-of-day, which updates maybe once per minute, so if you have hundreds or thousands or hundreds of thousands of requests per minute, this dynamic data is really static for one minute. A city's weather forcast doesn't change quite as often. As far as scalability goes, the discussion has focused on components. However, I would like to mention that on the high end, this really ends up being about architecture and protocol, or how can the components be separated and how can they communicate. If the components themselves do not have built in communication APIs, then they becomes inseparable from other components. It sounds like the nsdci module would extend the architecture of AOLserver significantly for anyone interested in scalablility. Maybe overall, the term scalability is something which applies to an application, not the components which are used to create the application. Components themselves might have an overall efficiency which measures how much time it takes to perform a given task. Eventually a given processor/thread/server will run out of resources or time. Scalability might be a measure of how easy it is to scale beyond a single instance of a component. tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.