Re: [AOLSERVER] Roadmap - 4.6 and beyond
As a contributor to aolserver and naviserver i want to add a few comments - we are running between 30 and 50 servers for various projects, i would say that 70% are naviserver right now. - the reason we switched from aolserver to naviserver was that with our load-pattern (using OpenACS) we experienced some problems: * up to about 1000 concurrent users aolserver was perfectly fine * above this, we saw crashes, running out of resources (connection threads), memory growth, etc., thread lockups, micro-lockups for a few seconds. Some of these lead to contributions to aolserver i did in the past * to pinpoint the problem we moved to Zoran's setup (tcl version, naviserver), that went though heaving testing on his side and was rock-stable * some of our problems disappeared/changed, some not (burst creation of theads,...). We have quite different load patterns than zoran. * we found sources for our problems at various places (server, tcl, ... machine architecture) depending as well on e.g. tcl versions etc. By now, most of the problems are gone, we are using NaviServer in production since more than two years. A summary is on the link referenced below. Even more recently, we exchanged the hardware to a more mainstream one (this improved the performance by a factor of 3-4!). The fact that e.g. the resource consumption went down, helped a lot to run on a much cheaper machine (memory consumption, max number of connection threads went from 80 to 30, etc.). Btw., in this process of moving from POWER to Intel apparently the biggest source of our crashes went away. The way Tcl handles thread-local storage (Tcl 8.5) seems to cause under heavy load race conditions, which lead to crashes in otherwise stable code-pieces (e.g. regexp). I rewrote some of the usages of the tls infrastructure in tcl to use GCC's non-standard tls handling via __thread, then the problem went from regexp to other places using tls). The problem was most likely dirty reads in tcl + mutex handling + POWER + gcc (from rhel). Tcl 8.6 is supposed to be better in this regard. For the changes in naviserver, see [1]. With the recent versions of naviserver/tcl 8.5/libthread the server runs in a stable memory size, without the need for daily reboots (although a reboot has some nice self-healing properties for nsvs, etc.). See [2] for a statistics from a machine with two naviservers with different configuration running (alice, nm). Among other things one can see the stable rss and vsizes of the servers over the last few months. Aolserver is in terms of memory leaks not so bad either. One can see on [3] the statistics from openacs.org and translate.openacs.org which is runing aolserver 4.5.1. One can see, where we fixed an application leak in May [4]. [1] https://bitbucket.org/naviserver/naviserver/src/5df3b1cb9ea6/NEWS [2] http://alice.wu.ac.at/munin/wu.ac.at/alice.wu.ac.at/index.html#naviserver [3] http://openacs.org/munin/localdomain/localhost.localdomain/index.html#naviserver [4] http://openacs.org/munin/localdomain/localhost.localdomain/naviserver_translate_memsize.html Concerning the comments below - the documentation of naviserver is at least par with aolserver (the man pages are quite good). - for me, the the biggest pain is the aolserver-naviserver config file conversion, but the actual documented config files on bitbucket contain now all values read from the server with the default values. - porting all the changes from naviserver into aolserver is much more work than the other way round. i have no problem with the coexistence of naviserver and aolserver, providing urgent changes to both servers (as i have done in the past). - both aolserver and naviserver are stable and mature (having advantages and disadvantages), the people running large sites are rather conservative. Having alternatives is rather a selling argument. If e.g. aolserver is dropping windows support, naviserver can continue it (or vice versa). -gustaf neumann On 27.09.12 23:25, John Buckman from BookMooch wrote: Naviserver has added a lot of interesting features, and appears to be fairly mature. I would have probably switched to Naviserver two years ago if they had documented some of their changes. The quantity of the contributions, and the interesting nature of many of them, make me feel that Naviserver is far from end of life. When I switched (temporarily) to naviserver I found enough things that didn't work like aolserver, yet were totally undocumented, that the experience was very frustrating and I went back to aolserver. I was spending too much time reading C source code to figure things out. So... my personal vote for an aolserver v5 would be merging in lots of the naviserver code changes into aolserver. There's a lot of bang-for-our-buck there. Or, simply running with naviserver, if we (the aolserver community) can get it
Re: [AOLSERVER] Roadmap - 4.6 and beyond
Has anyone analyzed Naviserver performance and features vs. AOLserver lately? It appears to remain compatible with Windows. The following forum post suggests Naviserver may be a contributing factor to a significant overall performance increase: http://openacs.org/forums/message-view?message_id=3957131 Maybe AOLserver 5 should start as a fork of Naviserver? ;-) On 09/27/2012 06:25 AM, Andrew Piskorski wrote: On Wed, Sep 26, 2012 at 09:07:07AM -0600, jgdavid...@mac.com wrote: Should we dump the Windows port in favor of a clean Unix code base, configure, build, and install? Cross-platform portability is very Nice to Have, and I've actually used it. Fortunately I've never had to deal with or even look at the Unix vs. Windows compatibility layer, so I can't speak on its maintenance burden. It just worked. That was really nice when (unexpectedly) porting my code from Solaris to Windows. The AOLserver Windows build process back then indeed seemed pretty bad; far too much unscriptable Microsoft project file crap. But I thought someone cleaned that up later. On Wed, Sep 26, 2012 at 09:24:41AM -0600, jgdavid...@mac.com wrote: For folks using Windows, I always follow up with the question: With VMware, Parallels, etc. today, even if you're bound to Windows hardware, can you just virtualize away the difference? Absolutely not. In my case, it's because I use AOLserver as a custom app server, which needs to build against proprietary libraries that are ONLY available on Windows. If what you need is a native Windows library, running Linux in a virtualized container is of no help whatsoever, you might as well be running on a remote Linux box. (Cygwin I'm not sure about. Can Cygwin applications build with native Windows libraries?) In my case, actually what happened is the proprietary vendor library discontinued Solaris support, leaving only Windows, so I ported to Windows. I'm still running it on Windows today. Now, if I was writing that same app today I might do things somewhat differently. But it was Really Nice that the custom AOLserver app I'd originally written on Solaris mostly Just Worked when I ported it to Windows. My own AOLserver on Windows use case is likely very atypical, but I think it's still a useful minor example of how cross-platform portability can be unexpectedly helpful, even when you originally didn't think you'd need or want it. Is cross-platform support in AOLserver worth the maintenance burden? Not having worked on that code, I can't say. But I can say that the cross-platform code does have real value, and was probably a lot of work to get right way back when, so I'd caution against throwing it out without a very clear case that doing so is worth more than the loss, and that there isn't some better way to achieve the same gains. My (completely unfounded in any hard evidence) gut-level suspicion is that 80% of the simplicity gains to be had from completely discarding Windows support are likely achievable by instead re-factoring (somehow or other) whatever parts are actually giving maintainers the most trouble. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ aolserver-talk mailing list aolserver-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aolserver-talk
Re: [AOLSERVER] Roadmap - 4.6 and beyond
Hi, Every few years we talk about what's next for the strategic direction of AOLserver which is great. In addition to the ideas below (which are cool), I always bring up this question: Should we dump the Windows port in favor of a clean Unix code base, configure, build, and install? I wrote most of that weird Windows code, including the goofy nsconfig stuff. Some of it was curious, maybe even clever, but in the end it was a distraction. It's impact on the config/build process in particular was pretty significant. Today's Linux and OS/X environments are so much more amenable to Aolserver, with threaded Tcl ready to go, gcc/make all pretty stable. It wasn't like that in the early days!For me, a purge of the Windows code and then an aggressive scan for anything still not 64-bit compatible and cleanly build-able using standard configure/gcc/gmake tools would be quite refreshing :) -Jim On Sep 26, 2012, at 7:47 AM, Cesáreo García Rodicio cesa...@cesareox.com wrote: Hi all, Firstly, thanks so much for your work. A lot of us are using aolserver everyday so this is welcome !! I'm not a hard developer but in my projects it's been hard students to install and use aolserver). And I think it's because documentation and installation: 1. TCL API and Config Files 2. Packaged Installation (batteries included) 3. Some Case Studies and Complete Examples with API (something simple). Only some ideas. Great Work! Cesáreo El 25/septiembre/12 05:29, Jeff Rogers escribió: Hi all, There should be a 4.5.2 final release sometime soon, but what comes next? I've been organizing my wishlist of what I'd like to see in future AOLserver releases and I'm throwing it out there for anyone else to add to or comment on. These are not in any particular order; some are half-baked, some are straightforward, and some are little more than speculation. I know development hands are a bit short these days, but maybe people will find something that interests them to work on. Core features: - support chunked postdata - api for filter unregistration - core async delivery currently possible by transferring conn socket to tcl event loop. Would be nice to make it work for everything, by default. - re-queue api extension of pre-queue filters and quewait api: allow a conn thread to send a request back to quewait for network i/o. - move encoding and compression to filters - general-purpose worker-pool api - external prebinding allow an external program to bind ports and specify open file descriptors on the command line; would allow privileged port binding with no root privileges for actual server. Would also allow restarting without closing listen socket. - pre-start request service have a micro server that responds to requests with please wait while server is starting. Helpful for long start-up sequences. Core tcl: - replace various c-coded file commands with tcl equivalents (e.g., ns_mkdir, ns_unlink). Main benefit is clean handling of utf8 filenames. - Support a 2-phase interp initialization. Phase 1 is defining procs / loading packages, which is replicated in every new interp. Phase 2 is initializing persistent data, preloading caches, setting up filters and handlers, etc; things that are not replicated in every new interp. Nsdb: - add variable binding to nsdb - add lob handling to nsdb - support runtime db pool configuration Protocols: - SPDY - websockets I have a vague notion of how both of these could work. But it needs somewhat more than that :) Documentation: - Yes, please. Packaging: - more config examples - examples of various features - configuration through web browser - batteries-included distribution (binaries including perhaps sqlite, zlib, openssl, a few simple web apps, maybe php, perl, ...?) - single-file mountable packages, like tclkits Community: - dogfood website It'd be really nice if aolserver.com actually ran on aolserver. It's hosted on sourceforge currently so probably not much chance of that as it stands, but who knows. Anything else to add? -J -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ aolserver-talk mailing list aolserver-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aolserver-talk -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond.
