Re: [AOLSERVER] Suggestions for a future aolserver
I have to agree this all makes a lot of sense, especially focusing on what OpenACS needs. Making that all batteries-included by integrating the most common items (OpenSSL, MySQL, Postgres) seems smart. -Jim Sent from my iPhone On Sep 28, 2012, at 14:17, Maurizio Martignano wrote: > Well, > I am reacting to the “salt on the open wound” and giving my > opinion. > > 1. Aolserver does what is supposed to do (i.e. a webserver with TCL > support) and does it well. > 2. When people are now talking about things like “node.js”, “Inria – > hop”, “Google GWT” this is not going back to provide some JS (or PHP) > support; here the idea is to have a single paradigm, a single programming > model client-side and server-side and this is rather new. Even if we added > these things, people would not leave their preferred mainstream platform, > just to switch to Aolserver. And where would be the energy to implement all > of this stuff? > 3. I believe, I might be wrong, that lots of people are not really > interested in Aolserver per se, they are interested in having the full > software stack: Aolserver + TCL + PostgreSQL + OpenACS working and working > well together. > 4. So what I would do is: > a. Aolserver - Make it stable versus nowadays available OSes( eg. add > the IPV6, if necessary, remove any 32 bit things if they are still there, > both in the Unix part and in the Windows part, and so on…), > b. OpenACS – update the data model so that instead of hierarchical > structures CTEs are used (this would allow OpenACS to work properly on > current versions on PostgreSQL, and Oracle too ) > c. OpenACS – Improve the user interfaces, with a particular eye to > mobile platforms (yes here a lot of JS will be required but it is not the > same thing as “node.js”, this is more the “traditional” JS) > d. Applications (that is the most difficult part) – they should change > their code base to use these new features (e.g. CTE versus hierarchical > structures). > > I am aware that I somehow “trespassed the Aolserver only borders” and I do > apologize for that, but I believe we cannot speak about Aolserver alone, nor > about OpenACS alone and so on… All the components stack must converge towards > the same future. > > Again sorry for the mumbo jumbo. > > Ciao, > Maurizio > > > From: Jim Davidson [mailto:jgdavid...@mac.com] > Sent: 28 September 2012 21:14 > To: Jeff Rogers > Cc: aolserver-talk@lists.sourceforge.net > Subject: Re: [AOLSERVER] Suggestions for a future aolserver > > > Howdy, > > The Digital City extensions were written way back in 1998 :) I wrote a big > chunk of the code along with Jay Ridgeway, Nate Folkman, Shaz Chaudhri and > others. We used it extensively for sites like Aol.com, MapQuest, Moviefone, > etc. It had a bunch of goodies, many pragmatic solutions to large-scale, > multi-server installations. One popular item was "sob" which pre-dates and > was similar in use to memcache but, assuming well cacheable objects, far > faster because it utilized AolServer's large shared-memory model and > persistent control connections for a a coherent, multi-server cache. I like > to say that today's memcache "hit" was our sob-based cache "miss". It was > typical for an nsdci-based AolServer to make dozens or 100's of nssob calls > building a single page within a few 100 milliseconds. There were some other > interesting pieces like "av" which was a pre-sorted key-value pair file that > would be distributed out to the servers (could be 100 servers or more) and > memmapped in "instantly". This was the basis for a huge chunk of > pre-computed published content, e.g., promotion indices and such. I don't > think any of it is still in use at Aol today -- for various reasons new > technologies are now used but I haven't heard that anything new approached > the same level of efficiency and performance. > > The downside to that stuff was topology management -- with 100 or more > connected servers for Aol.com, for example, managing the Tcl-based config > files went from something clever or at least curious to despised.Some of > the over-reliance on pre-computation of data structures in the interest of > ultimate run-time performance also began to wear on the patience of editors > who had to use increasingly over burdened back-office publishing systems > (Aolserver based as well). > So, all things in moderation, everything has it's day in the sun :) > > Having said that, the nsdci module is very cool if a bit undocumented and > arcane on the config side. Oh, and tha
Re: [AOLSERVER] Suggestions for a future aolserver
Well, I am reacting to the salt on the open wound and giving my opinion. 1. Aolserver does what is supposed to do (i.e. a webserver with TCL support) and does it well. 2. When people are now talking about things like node.js, Inria hop, Google GWT this is not going back to provide some JS (or PHP) support; here the idea is to have a single paradigm, a single programming model client-side and server-side and this is rather new. Even if we added these things, people would not leave their preferred mainstream platform, just to switch to Aolserver. And where would be the energy to implement all of this stuff? 3. I believe, I might be wrong, that lots of people are not really interested in Aolserver per se, they are interested in having the full software stack: Aolserver + TCL + PostgreSQL + OpenACS working and working well together. 4. So what I would do is: a. Aolserver - Make it stable versus nowadays available OSes( eg. add the IPV6, if necessary, remove any 32 bit things if they are still there, both in the Unix part and in the Windows part, and so on ), b. OpenACS update the data model so that instead of hierarchical structures CTEs are used (this would allow OpenACS to work properly on current versions on PostgreSQL, and Oracle too ) c. OpenACS Improve the user interfaces, with a particular eye to mobile platforms (yes here a lot of JS will be required but it is not the same thing as node.js, this is more the traditional JS) d. Applications (that is the most difficult part) they should change their code base to use these new features (e.g. CTE versus hierarchical structures). I am aware that I somehow trespassed the Aolserver only borders and I do apologize for that, but I believe we cannot speak about Aolserver alone, nor about OpenACS alone and so on All the components stack must converge towards the same future. Again sorry for the mumbo jumbo. Ciao, Maurizio From: Jim Davidson [mailto:jgdavid...@mac.com] Sent: 28 September 2012 21:14 To: Jeff Rogers Cc: aolserver-talk@lists.sourceforge.net Subject: Re: [AOLSERVER] Suggestions for a future aolserver Howdy, The Digital City extensions were written way back in 1998 :) I wrote a big chunk of the code along with Jay Ridgeway, Nate Folkman, Shaz Chaudhri and others. We used it extensively for sites like Aol.com <http://aol.com/> , MapQuest, Moviefone, etc. It had a bunch of goodies, many pragmatic solutions to large-scale, multi-server installations. One popular item was "sob" which pre-dates and was similar in use to memcache but, assuming well cacheable objects, far faster because it utilized AolServer's large shared-memory model and persistent control connections for a a coherent, multi-server cache. I like to say that today's memcache "hit" was our sob-based cache "miss". It was typical for an nsdci-based AolServer to make dozens or 100's of nssob calls building a single page within a few 100 milliseconds. There were some other interesting pieces like "av" which was a pre-sorted key-value pair file that would be distributed out to the servers (could be 100 servers or more) and memmapped in "instantly". This was the basis for a huge chunk of pre-computed published content, e.g., promotion indices and such. I don't think any of it is still in use at Aol today -- for various reasons new technologies are now used but I haven't heard that anything new approached the same level of efficiency and performance. The downside to that stuff was topology management -- with 100 or more connected servers for Aol.com <http://aol.com/> , for example, managing the Tcl-based config files went from something clever or at least curious to despised.Some of the over-reliance on pre-computation of data structures in the interest of ultimate run-time performance also began to wear on the patience of editors who had to use increasingly over burdened back-office publishing systems (Aolserver based as well). So, all things in moderation, everything has it's day in the sun :) Having said that, the nsdci module is very cool if a bit undocumented and arcane on the config side. Oh, and that av thing? It's not 64-bit safe -- the offsets and lengths as 32-bit integers in the on-disk file were swapped to direct pointers on load. Didn't seem like a problem back in 1999 Anyway, from what I've heard about node.js (which was just a guy giving a talk about it at a conference), it sounds very cool. I think getting javascript and/or php to run well with Aolserver would be interesting. Tcl would always have a special place and any implementation would need a "call Tcl..." thinger, but sounds like a cool investigation. Just to throw salt on the open wound do folks want stuff like node.js or Windows support :) Just asking..
Re: [AOLSERVER] Suggestions for a future aolserver
Howdy, The Digital City extensions were written way back in 1998 :) I wrote a big chunk of the code along with Jay Ridgeway, Nate Folkman, Shaz Chaudhri and others. We used it extensively for sites like Aol.com, MapQuest, Moviefone, etc. It had a bunch of goodies, many pragmatic solutions to large-scale, multi-server installations. One popular item was "sob" which pre-dates and was similar in use to memcache but, assuming well cacheable objects, far faster because it utilized AolServer's large shared-memory model and persistent control connections for a a coherent, multi-server cache. I like to say that today's memcache "hit" was our sob-based cache "miss". It was typical for an nsdci-based AolServer to make dozens or 100's of nssob calls building a single page within a few 100 milliseconds. There were some other interesting pieces like "av" which was a pre-sorted key-value pair file that would be distributed out to the servers (could be 100 servers or more) and memmapped in "instantly". This was the basis for a huge chunk of pre-computed published content, e.g., promotion indices and such. I don't think any of it is still in use at Aol today -- for various reasons new technologies are now used but I haven't heard that anything new approached the same level of efficiency and performance. The downside to that stuff was topology management -- with 100 or more connected servers for Aol.com, for example, managing the Tcl-based config files went from something clever or at least curious to despised.Some of the over-reliance on pre-computation of data structures in the interest of ultimate run-time performance also began to wear on the patience of editors who had to use increasingly over burdened back-office publishing systems (Aolserver based as well). So, all things in moderation, everything has it's day in the sun :) Having said that, the nsdci module is very cool if a bit undocumented and arcane on the config side. Oh, and that av thing? It's not 64-bit safe -- the offsets and lengths as 32-bit integers in the on-disk file were swapped to direct pointers on load. Didn't seem like a problem back in 1999 Anyway, from what I've heard about node.js (which was just a guy giving a talk about it at a conference), it sounds very cool. I think getting javascript and/or php to run well with Aolserver would be interesting. Tcl would always have a special place and any implementation would need a "call Tcl..." thinger, but sounds like a cool investigation. Just to throw salt on the open wound do folks want stuff like node.js or Windows support :) Just asking... Cheers, -Jim On Sep 27, 2012, at 7:41 PM, Jeff Rogers wrote: > John Buckman wrote: >> SUGGESTIONS FOR A FUTURE AOLSERVER >> >> The "state of the art" is, I think, happening in the javascript world, >> with things such as node.js. If the aolserver community were really >> interested in getting new users, making it a top notch >> embedded-javascript web server would be a way. I'm not sure this makes >> any strategic sense (nor that I need it), but there you go... > > I'm far from the first person to chuckle at how netscape introduced > server-side javascript 15 years ago, it pretty much vanished for a > while, and has now come roaring back as the next big thing. > > Plus ça change, plus c'est la même chose. > > Still, a "ns_spidermonkey" module would be a interesting project. > >> The other area where "state of the art" thinking is occurring, is in >> scaling web sites to many, many machines. >> >> aolserver has some features to make that happen, but imagine if we had >> multi-machine ns_cache support, perhaps with a file system backup (ie, >> memcached)? > > I was thinking about this also. There is a nsmemcache module for > naviserver, but it uses some of the C apis that have been changed > slightly in naviserver, so it doesn't work with aolserver, but it > shouldn't be difficult to port. > > A version (not sure if it's the most current) is at > http://naviserver.cvs.sourceforge.net/naviserver/modules/nsmemcache/ > > Another bit of existing code to build a high-scalability cluster is the > Digital City extensions that the guys at AOL built sometime before 2006 > (I don't know exactly when, just estimating based on timestamps) and I > believe ran some pretty big (for the day) sites on. It looks like a > hidden treasure of functionality that never got the respect it deserved. > Looking at the archives, there was some chatter about it on this list > back in 2007. What's that french saying again? :) > > It currently lives at http://code.google.com/p/nsdci/ > >> Also, support for sending an http page request to another machine (or >> pool of machines) would very handy, with a single http daemon handling >> them all with async io. > > I don't think this would be difficult as things are now; it would make > a nice piece of example code. > > -J > > --
Re: [AOLSERVER] Suggestions for a future aolserver
On Thu, Sep 27, 2012 at 02:44:16PM -0700, John Buckman wrote: > The other area where "state of the art" thinking is occurring, is in > scaling web sites to many, many machines. Reputedly, the best toolkit for building that sort of stuff is Erlang and its OTP libraries. (Asynchronous message passing, hot code reloading, fault tolerance, etc.) I've also long wondered if its Mnesia distributed RDBMS is any good. I was quite happy with AOLserver, Tcl, and a decent single-box RDBMS like Oracle or PostgreSQL. But if I intended to build stuff for massive scale out, ideally I'd want to first hack seriously with Erlang for a year or so first to really understand what it's good for, what it's not, clever approaches the Erlang community came up with, etc. If anybody here as done something along those lines and can report back, compare/contrast to the AOLserver / Tcl / OpenACS world, that could be really interesting! A maybe related approach from the "enterprise" software world (probably meaning giant investment banks), is Message-Oriented Middleware; this is also asynchronous message passing. This guy (Kirk Wylie, founder of OpenGamma) seems to know what he's talking about, and recommended the Hohpe book below: http://kirkwylie.blogspot.com/ http://www.amazon.com/Enterprise-Integration-Patterns-Designing-Deploying/dp/0321200683/ Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe, Bobby Woolf > Ousterhout recently wrote a paper about RAMCloud, which would be very helpful > on aolserver: > http://cacm.acm.org/magazines/2011/7/109885-the-case-for-ramcloud/fulltext Useful I guess, but that seems pretty low level. I'd rather look into shared-nothing parallel RDBMSs. These were mostly intended for analytic (date warehouse) loads, but there are OLTP-oriented designs available now too (e.g. VoltDB, which also happens to be RAM-only). I don't know how well they scale across modes. Years ago, I once heard that, Teradata was only intended to scale to no more than 100 or so fat nodes. But even if modern shared-nothing OLTP systems scale no better, that's still about 100 times better than you could do with a typical single server Oracle or PostgreSQL installation, which should give you a lot of helpful leeway before you HAVE to develop and adopt database sharding and/or more specialized tools, like the Digital City team did. The recent Calvin OLTP research is also interesting. Like "RamCloud", this is a lower-level tool, not a complete system. Unlike RamCloud though, it seems like an actual tested advance in the state-of-the art, not just a conceptual description of a cool piece of infrastructure someone might want to build: http://dbmsmusings.blogspot.com/2012/05/if-all-these-new-dbms-technologies-are.html -- Andrew Piskorski -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ aolserver-talk mailing list aolserver-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aolserver-talk
Re: [AOLSERVER] Suggestions for a future aolserver
> I was thinking about this also. There is a nsmemcache module for > naviserver, but it uses some of the C apis that have been changed > slightly in naviserver, so it doesn't work with aolserver, but it > shouldn't be difficult to port. memcached is lovely, but what makes ns_cache so fast is that it runs in-process as a function call, and thus suffers no networking overhead. If it were database (or dbm) backed, that'd be great. What I was thinking was using ns_cache on your own machine, but with a background "sync" thread that informed other aolserver instances of changes, so the various aolserve's ns_caches stayed in sync. That'd be small and compact, and fairly foolproof. I currently use BerkeleyDB for this on bookmooch, but the multi-machine syncing in bdb is very heavyweight and confusing as all hell, but it is at least peer-to-peer, so new aolserver instances can come die or come up, and in theory it all works. But.. the zero-latency of a function call cache lookup (vs memcached) is so crazy good that it makes it worth it. With MoodMixes, I use ns_cache to do several hundred lookups per html page, when running pages in non-english (ie, each phrase is looked up). The network latency of a network-based cache wouldn't really work, unless I go the much-more-complicated route of massive async concurrent lookups. The dci stuff appears to be nicely thought out, but personally I wouldn't use it, as all the cache operations appear to be client/server, and thus incur a latency overhead. -john -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ aolserver-talk mailing list aolserver-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aolserver-talk
Re: [AOLSERVER] Suggestions for a future aolserver
John Buckman wrote: > SUGGESTIONS FOR A FUTURE AOLSERVER > > The "state of the art" is, I think, happening in the javascript world, > with things such as node.js. If the aolserver community were really > interested in getting new users, making it a top notch > embedded-javascript web server would be a way. I'm not sure this makes > any strategic sense (nor that I need it), but there you go... I'm far from the first person to chuckle at how netscape introduced server-side javascript 15 years ago, it pretty much vanished for a while, and has now come roaring back as the next big thing. Plus ça change, plus c'est la même chose. Still, a "ns_spidermonkey" module would be a interesting project. > The other area where "state of the art" thinking is occurring, is in > scaling web sites to many, many machines. > > aolserver has some features to make that happen, but imagine if we had > multi-machine ns_cache support, perhaps with a file system backup (ie, > memcached)? I was thinking about this also. There is a nsmemcache module for naviserver, but it uses some of the C apis that have been changed slightly in naviserver, so it doesn't work with aolserver, but it shouldn't be difficult to port. A version (not sure if it's the most current) is at http://naviserver.cvs.sourceforge.net/naviserver/modules/nsmemcache/ Another bit of existing code to build a high-scalability cluster is the Digital City extensions that the guys at AOL built sometime before 2006 (I don't know exactly when, just estimating based on timestamps) and I believe ran some pretty big (for the day) sites on. It looks like a hidden treasure of functionality that never got the respect it deserved. Looking at the archives, there was some chatter about it on this list back in 2007. What's that french saying again? :) It currently lives at http://code.google.com/p/nsdci/ > Also, support for sending an http page request to another machine (or > pool of machines) would very handy, with a single http daemon handling > them all with async io. I don't think this would be difficult as things are now; it would make a nice piece of example code. -J -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ aolserver-talk mailing list aolserver-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aolserver-talk
