Re: [AOLSERVER] AOLserver and Tcl Ports to iPhone
Yeah, it was an interesting idea... ;) http://www.freshpatents.com/Client-server-web-application-architectures-for-offline-usage-data-structures-and-related-methods-dt20070208ptan20070033569.php - n On 10/31/07, patrick o'leary [EMAIL PROTECTED] wrote: It's funny, when I used work for Eric, every time our operations / sysadmin group complained about disk usage cost of hardware, we'd say we'd work on porting aolserver to the iPod, and slowly start replacing boxes with cheap iPods Emmm it was a joke Eric, just a joke. ;-) But I've actually seen a few apps developed by running aolserver on the desktop, instead of having to learn the rc components of each operating system, your display mechanism can simply be html. Google's original desktop search app was a light webserver, so aolsever folks weren't the only ones to think of it. p Tomasz Kosiak wrote: FYI: Eric O'Laughlen ported AOLserver and Tcl to iPhone http://velocipeek.com/2007/09/17/iphone-aolserver-and-tcl-ports/ --tkosiak Here is a quote from his blog = http://velocipeek.com/2007/09/21/joggame-server/ Friday, September 21st, 2007 at 5:49 am Joggame Server By eolaughlen Song Of The Day: Beautiful Day Artist: U2 Today, I am pleased to announce a spin-off of the open source AOLserver project for devices: Joggame Server (pronounced jog-gu-mE). The technology, acquired by AOL in 1995 from NaviSoft, has a long history of technical achievement and innovation. It was one of the first to offer multi-threaded support. It was one of the first to embed an interpreter versus using the slower, and less efficient, CGI. For the iPhone it is one of the first web application development environments. Its ease-of-use, and long history of reliability, security, and cross-platform support, make it the perfect choice for the iPhone and other web-enabled devices. With this new vision my hope is that the Joggame Server project will continue to carry the legacy forward. == -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Patrick O'Leary AOL Local Search Technologies Phone: + 1 703 265 8763 You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles. Do you understand this? And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat. - Albert Einstein [image: View Patrick O Leary's LinkedIn profile]View Patrick O Leary's profile http://www.linkedin.com/in/pjaol -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. btn_in_20x15.gif
Re: [AOLSERVER] Large file uploads being limited
Here's an example showing how to find what is currently configured, as well as how to update the maxupload value. You can reset the value directly in your AOLserver configuration file if you'd like. Welcome to server1 running at /usr/local/aolserver/bin/nsd (pid 570) AOLserver/4.5.0 (aolserver4_5) for osx built on May 11 2007 at 17:19:41 CVS Tag: $Name: $ server1:nscp 1 ns_limits list default server1:nscp 2 ns_limits get default nrunning 0 nwaiting 0 ntimeout 0 ndropped 0 noverflow 0 maxwait 100 maxupload 1024 timeout 60 maxrun 100 server1:nscp 3 ns_limits set default -maxupload [expr 1024 * 2] nrunning 0 nwaiting 0 ntimeout 0 ndropped 0 noverflow 0 maxwait 100 maxupload 2048 timeout 60 maxrun 100 Hope that helps! - n On 8/10/07, Dossy Shiobara [EMAIL PROTECTED] wrote: On 2007.08.10, Pedro Liska [EMAIL PROTECTED] wrote: I'm using AOLserver 4.5.0 with nsopenssl v3_0beta26 . When I post a file that is greater than 10MB to the server, I get a The connection was reset error in Mozilla and a Internet Explorer cannot display the webpage error in IE7. Connection-oriented limits are now specified with the ns_limit command in 4.5.0 -- here's the snippet from the RELEASE file: ns_limit: The ns_limit command enables setting various resource limits for specified method/url combinations. These limits include such items as max concurrent connections, max file upload size, and timeouts waiting for connection processing. When limits are exceeded, connections are immediately dispatched to a dedicated error connection processing pool to generate a quick error response. By default all requests share the same default limits. Coupled with the new ns_pools command, URL-based limits can provide for sophisticated resource management. -- Dossy -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Speaking of nsdci...
On 8/9/07, John Buckman [EMAIL PROTECTED] wrote: On Aug 9, 2007, at 2:07 AM, Nathan Folkman wrote: A variant is used in production, although it's not exactly the same. Can you say more about variant? The biggest difference is that the version in Google Code includes support for using HTTP as a transport in addition to the persistent DCI_RPC transport (see rpc.c). There's only a couple of us working on updating this code at the moment, but I'd be more then happy to add anyone to the project who is interested in contributing. There are a bunch of sample configuration files, which honestly is the most confusing part of this code base. Contact me off list if interested in pitching in, and I'll get you set up. Ok, so it's messy right now, so probably not good for merging in. It's actually not that bad - could always be better of course! ;) Anyhow, I'm not that concerned about this feature right now, but Dossy mentioned wanting to know why I used lighthttpd for large files instead of aolserver, and that's how this async issue came up. Not sure if this module would help you with large files, but AOLserver 4.5might given the changes made to how the driver thread interacts with the pool of connection threads. Suggest running some tests. - n -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Speaking of nsdci...
It hasn't been officially released or announced as of yet. Tt's in more of an initial development release phase at the moment. Still lots of documentation to be done. There is currently no roadmap or timeline to share with everyone either at this time. - n On 8/8/07, Rick Cobb [EMAIL PROTECTED] wrote: I see it on Google Code (http://code.google.com/p/nsdci/), but not on the modules wiki (http://panoptic.com/wiki/aolserver/Modules#Miscellaneous); I'll add it, but wondered why it wasn't there? What's its status? -- ReC -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Speaking of nsdci...
A variant is used in production, although it's not exactly the same. There's only a couple of us working on updating this code at the moment, but I'd be more then happy to add anyone to the project who is interested in contributing. There are a bunch of sample configuration files, which honestly is the most confusing part of this code base. Contact me off list if interested in pitching in, and I'll get you set up. - n On 8/8/07, Jade Rubick [EMAIL PROTECTED] wrote: Is it used in production? Is there a plan to do any further development or documentation for it? Jade On 8/8/07, Nathan Folkman [EMAIL PROTECTED] wrote: It hasn't been officially released or announced as of yet. Tt's in more of an initial development release phase at the moment. Still lots of documentation to be done. There is currently no roadmap or timeline to share with everyone either at this time. - n On 8/8/07, Rick Cobb [EMAIL PROTECTED] wrote: I see it on Google Code ( http://code.google.com/p/nsdci/), but not on the modules wiki (http://panoptic.com/wiki/aolserver/Modules#Miscellaneous ); I'll add it, but wondered why it wasn't there? What's its status? -- ReC -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Jade Rubick Senior Developer United eWay Volunteer Solutions [EMAIL PROTECTED] tel (503)285-4963 fax (707)671-1333 www.UNITEDeWAY.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver focus
from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver focus
It is possible. ;) There are a number of other things you can do as well to help. One example is configuring your threads such that they die and get reaped after a certain number of requests or time. This will cause memory that is currently tied up in the thread's memory pool to be returned back to the shared memory pool, which can have the net effect of lowering the overall process size. - n On 8/7/07, Daniël Mantione [EMAIL PROTECTED] wrote: Op Tue, 7 Aug 2007, schreef Nathan Folkman: You might also want to try running AOLserver without the Tcl threaded allocator (Zippy). You might want to try Hoard or if on Linux maybe give Google's TCMalloc a shot. Remember, the Zippy allocator is optimized for lock avoidance, and this comes at the cost of greater memory overhead. Yup, I have installations where AOLserver uses very little memory. Try this with Apache :) wwwrun4552 0.0 1.4 7380 3612 ?Sl Jul26 2:21 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun Daniël -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] learning from naviserver
There's a lot of really innovative stuff going on with the naviserver project, and they've done a lot of work to stay in sync with changes being made to the core AOLserver project. I suggest folks check it out. - n On 8/3/07, John Buckman [EMAIL PROTECTED] wrote: So, in the spirit of open source software meritocracies: please place your money where your mouth is. Come up with a list of actionable changes you'd make if you were king. Lets hear it--and if everyone agrees to a particular change, we'll declare it made. (Note: declaring anything to a volunteer-driven organization doesn't guarantee that anyone will actually do it.) In no way do I want to be king, and in an effort to calm things down a bit, let me say that I'm really, really happy with aolserver, and the last major release had amazing things in it. My only real nit is that so many cool things are not-so-well-documented, but hell, it's open source. I just spot things in the text files, and then figure it out by reading the C code, which is often commented, but always clean and readable. On the subject of cool things and not-so-well-documented, I'd like to bring up Naviserver (you can find it on sourceforge, it's an independent fork of aolsever, at http://naviserver.sourceforge.net). Their fork of aolserver has an insane number of changes to it, and lots of great ideas (look at http://naviserver.cvs.sf.net/naviserver/ naviserver/ChangeLog?view=markup). There's a handful of developers working on it, and it seems like a real hotbed of innovation. I've not used naviserver, though I evaluated it seriously, because it has so many differences to aolserver, almost all undocumented, that it was really hard for me to get up to speed to, and I found it less reliable than aolserver, probably because of all those innovations. I kind of like the slower pace of aolserver, I can actually run a production web site on it. This is not meant as an insult to Vlad and the other Naviserver developers, I'm just pointing out how the two development communities differ. However, I was wondering if we should perhaps look at merging back some of the best things in naviserver, back into aolserver. In fact, maybe we should treat the aolserver/naviserver split like ubuntu treats its two releases, and recognize that naviserver as an innovative, highly chaotic playground, and merging back in the best ideas from it back into aolserver after a long delay (6 months to a year) once each feature has settled down a bit and we can evaluate whether, in hindsight, it really was a good idea and the way it was implemented turned out well. I'd be game to go through the naviserver changelog in the future, and be part of a discussion of what is in there that we might want to merge back into aolserver. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
I'm someone who can, and will, remove your subscription from this list. - n On 8/3/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Please watch the vulgar language - there's simply no need for it. And precisely who are you to say so? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
We actually also tried a similar process a few years ago, but it failed due to various conflicts and general lack of interest. Would be great to get something like that going again. - n On 8/3/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: The reason why I reach out to organizations privately is because some of them are building commercial products on top of AOLserver and may not be comfortable discussing their needs in a public forum. Well, the way we handle such situations in the openacs community is that changes go through an approval process which we've copied from the Tcl crew. So these things CAN'T happen in private. The motivation - or the identity of a paying customer - can be kept private, of course, but changes in core code that changes functionality must be approved by core developers in a PUBLIC PROCESS. The developer community around AOLserver is small, so informality's fine IMO, but transparency is crucial. And BTW, I recognize that you've communicated with the OpenACS community as well as various companies, and that's a good thing. Don, what would you change to improve things? Of those suggestions, which could you personally do to help the project? Unfortunately my volunteer time is still allocated to OpenACS, so I can't really offer to help on a sustained basis, though I did drop in and implement virtual server awareness in ns_cache back when virtual servers were put in. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
There was a lot of discussion internally about whether or not to support virtual servers going forward. The main impetus was to try and further simplify the code base as much as possible. Virtual servers, while useful to many, add a great deal of complexity to the code. - n On 8/2/07, Tom Jackson [EMAIL PROTECTED] wrote: On Wednesday 01 August 2007 23:53, [EMAIL PROTECTED] wrote: Maybe this is the intent, although it would be easy to add the server name to the key. ISTM that virtual servers weren't considered when this code was written ... All code in current versions should be fully aware of virtual servers. The previous code in 4.0.10 was in urlspace.c and queue.c and was fully virtual server aware. The current version simply ditches this for no good reason, and by that I mean that the map storage and search are fully virtual server aware, but the threadpool creation code is a simple hash table keyed to the name of the threadpool. It also appears that on startup, two threadpools are created 'default' and 'error'. These are shared across virtual servers. Any redefinition of these affects all virtual servers. So question: we now have interps separate from threads, but what about memory, code, etc. Is this part of the interp or does it hang out with the thread? If the thread is just a clean slate, at least it is safe to reuse in another virtual server, otherwise this is a big bend over routine. But then we are left with the other option: threads pick up interps and run anywhere and everywhere on the virtual server and the interp eventually grows in size. Even if threads exit with given timeout, do interps? This negates the reason for threadpools entirely. This really needs to be discussed because the programming model could be disasterously affected and complicated by these decisions. tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
There was a desire to be able to tune some of the server parameters while the server was running in order to avoid certain cases where a restart would result in the loss of various caches which had already been warmed up. Admittedly this requirement came from requirements for various AOL properties, but most of the new features in 4.5 came about this way. - n On 8/2/07, Dossy Shiobara [EMAIL PROTECTED] wrote: On 2007.08.01, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: One of the hallmarks of AOLserver has always been ease of configuration. My sense is that we're drifting away from that? Frankly, one of the biggest FAQ's is how do I tune AOLserver? IMHO, ease of configuration is less to manually tune which means settings need to be able to be adjusted at runtime. Seriously, what value does being able to set max connections manually really offer? What people really want is a server that uses resources efficiently but in a way which doesn't run itself out of resources and starve itself. Rarely do I actually hear people say yes, I really want to limit the number of connections but rather I want to allow as many connections as my hardware can handle--how high should I set max connections? AOLserver is going to become even easier to set up and configure. These are growing pains, unfortunately. -- Dossy -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
You are absolutely correct. Lack of documentation continues to be one of the biggest issues with this project in my opinion. Not sure how best to resolve this at this point to be completely honest. Would be a great start if other folks would start contributing to the example configuration files and the man pages. Is there anything that's making this difficult, and preventing people from helping out? - n On 8/1/07, Ian Harding [EMAIL PROTECTED] wrote: Better I think to add them (commented out if need be) the default config file that ships with 4.5. I searched the documentation and couldn't find any mention of the new setup (until someone pointed it out) but plenty of talk about the old ones. I think it's in the release notes, but nobody reads those ;^) - Ian On 7/30/07, Jeff Rogers [EMAIL PROTECTED] wrote: Nathan Folkman wrote: Those parameters moved and are now controlled via the ns_pools Tcl command: This issue keeps coming up. How hard would it be to add in (for the next point release) compatibility code in init.tcl to parse the old-style parameters? -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Yes and no. ;) Technically everything in the configuration file is a Tcl command (ns_section, ns_param, etc.) so it's really not that much of a stretch. But I agree, it is different. Here's the deal, a decision was made a while back to try some new things with AOLserver 4.5, some of which we knew wouldn't be backwards compatible. Thread pools and limits are two pretty radical changes that were made to the core. Had we ever actually gotten around to deploying AOLserver 4.5 internally here at AOL, I'm sure we would have at least done the work to provide some backwards compatibility wrappers for things like this, but as yet we haven't. I'll try and update the default configuration file with some examples soon. Sorry for the confusion. - n On 8/1/07, Tom Jackson [EMAIL PROTECTED] wrote: Nathan, This has to be the most bizzar change to the configuration setup for AOLserver, is it really true? Now you have to execute commands inside the config file to set this? Since some have called for examples, would it be possible for the author of these changes to provide a few. I have been using the threadpool configuration as shown in this example (start at server.tcl, then source threadpool files): From: http://rmadilo.com/m2/servers/rmadilo/config/ # threadpool-default.tcl # Single Threadpool # pool = default ns_section ns/server/${server}/pool/$pool ns_param maxconnections100 ns_param minthreads4 ;# 0 ns_param maxthreads10 ns_param threadtimeout 120 ns_param map GET / ns_param map POST / tom jackson On Thursday 26 July 2007 08:53, Nathan Folkman wrote: You'd actually want to do it by adding the following to the end of your configuration file: ns_pools set procsmsgmgr -maxconns 100 -maxthreads 20 -minthreads 10 -timeout 10 ns_pools register procsmsgmgr server1 POST /proc/msgmgr You can then verify that everything worked via the AOLserver control port: server1:nscp 1 ns_pools list procsmsgmgr default error server1:nscp 2 ns_server threads procsmsgmgr {min 10} {max 20} {current 10} {idle 10} {stopping 0} server1:nscp 3 ns_pools get procsmsgmgr minthreads 10 maxthreads 20 idle 10 current 10 maxconns 100 queued 0 timeout 10 Hope that helps! - n On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Thanks Nathan for the info. I'm trying to configure a pool but ns_server threads procmsgmgr returns error on server log. [26/Jul/2007:18:06:07][4968.3074280352][-conn:0-] Error: Tcl exception: no such pool: procmsgmgr while executing ns_server threads procmsgmgr # ns_section ns/server/${servername}/pools ns_param procmsgmgr Message Manager Receiving Pool ns_section ns/server/${servername}/pool/procmsgmgr ns_param map {POST /proc/msgmgr} ns_param maxconnections 100 ns_param maxdropped 0 ns_param maxthreads 20 ns_param minthreads 10 ns_param threadtimeout60 Can you tell if my configuration is correct? regards, On 7/26/07, Nathan Folkman [EMAIL PROTECTED] wrote: Those parameters moved and are now controlled via the ns_pools Tcl command: ns_pools: The ns_pools command enables configuration of one or more pools of connection processing threads. The pools allow certain requests to be handled by specific threads. This could, for example, ensure multiple long running requests don't block other short running requests. Pools are selected based on method/url pairs similar to the mappings managed by the ns_register_proc command. By default, all requests are handled by a single, unlimited, default pool. There is also an error pool as described below. Coupled with the new ns_limits command, pools can provide for sophisticated resource management. See also the new ns_limits command: ns_limit: The ns_limit command enables setting various resource limits for specified method/url combinations. These limits include such items as max concurrent connections, max file upload size, and timeouts waiting for connection processing. When limits are exceeded, connections are immediately dispatched to a dedicated error connection processing pool to generate a quick error response. By default all requests share the same default limits. Coupled with the new ns_pools command, URL-based limits can provide for sophisticated resource management. On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Its 4.5.0 On 7/26/07, Nathan Folkman [EMAIL PROTECTED] wrote: What version of AOLserver are you running? On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Hi
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Yep, the documentationn or lack there of, continues to dog us. Did you read the release notes at least? It doesn't specifically mention this incompatibility, but does contain a lot of useful information. On 8/1/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: You are absolutely correct. Lack of documentation continues to be one of the biggest issues with this project in my opinion. Not sure how best to resolve this at this point to be completely honest. Would be a great start if other folks would start contributing to the example configuration files and the man pages. Is there anything that's making this difficult, and preventing people from helping out? So, if I'm not mistaken, not only were existing config files broken, the basic philosophy of how AOLserver is configured changed for this important case (dynamic proc call), but ... There's not even any documentation telling people this? Something this significant shouldn't be implemented without documentation, and frankly, it's the developer's responsibility to make sure it's done. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
What about just contributing documentation? What is making that hard? On 8/1/07, Michael Andrews [EMAIL PROTECTED] wrote: I think lack of process is a big deterrent. How do changes get rolled into releases, what are the coding standards, is there a review process, etc. On Aug 1, 2007, at 10:29 AM, Nathan Folkman wrote: You are absolutely correct. Lack of documentation continues to be one of the biggest issues with this project in my opinion. Not sure how best to resolve this at this point to be completely honest. Would be a great start if other folks would start contributing to the example configuration files and the man pages. Is there anything that's making this difficult, and preventing people from helping out? - n On 8/1/07, Ian Harding [EMAIL PROTECTED] wrote: Better I think to add them (commented out if need be) the default config file that ships with 4.5. I searched the documentation and couldn't find any mention of the new setup (until someone pointed it out) but plenty of talk about the old ones. I think it's in the release notes, but nobody reads those ;^) - Ian On 7/30/07, Jeff Rogers [EMAIL PROTECTED] wrote: Nathan Folkman wrote: Those parameters moved and are now controlled via the ns_pools Tcl command: This issue keeps coming up. How hard would it be to add in (for the next point release) compatibility code in init.tcl to parse the old-style parameters? -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
As I said before, if you have issues with the changes made in 4.5, simply do not upgrade. There were a number of factors that led to our decision to release as we did, when we did. Unfortunately a lot of the backwards compatibility work never got completed. Sorry. On 8/1/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Technically everything in the configuration file is a Tcl command (ns_section, ns_param, etc.) so it's really not that much of a stretch. But I agree, it is different. Technically, the configuration file is a bucket of bits, but that's not a very useful observation. Had we ever actually gotten around to deploying AOLserver 4.5 internally here at AOL, I'm sure we would have at least done the work to provide some backwards compatibility wrappers for things like this, but as yet we haven't. Geezus. In this case, the code simply shouldn't've gone in. Or it should've been offered to the community in its incomplete form, with the offer to incorporate it in 4.5 if volunteers in the community completed it. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Simple answer - don't upgrade to 4.5. As I was trying to explain before, we knew some things in 4.5 would not be backwards compatible. On 8/1/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Nathan, This has to be the most bizzar change to the configuration setup for AOLserver, is it really true? Now you have to execute commands inside the config file to set this? This is absolutely crazy. The init file has never required dynamic execution of procs to work, and to have something this basic as an exception is nuts. Not to mention that breaking every bloody site's config file in the process is rather rude. Some of us still use AOLserver for production sites, hard as that is to believe. Was there community discussion of this change before it was implemented? It represents a huge change in configuration philosophy. Who's bright idea was this? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
I think we did actually talk about some of the stuff that was coming down the pipe when we used to have those AOLserver chats. Not sure what happened to those. I personally haven't been officially assigned to AOLserver support for a while now. - n On 8/1/07, Nathan Folkman [EMAIL PROTECTED] wrote: To be honest and blunt, the community process in terms of AOLserver is rather broken, and has been for some time. The 4.5 work was done in-house here at AOL, and with limited community involvement. A lot of the experimental features were added to deal with specific scaling challenges we knew we would be facing here. When decided to go ahead and release 4.5, the thought was that we would eventually finish up any backwards compatibility work that needed to be done. We also assumed that if there were issues, people would simply hold off on upgrading, or work around the issues. Would documentation have helped - of course. Would a roadmap be useful - definitely. Agree with everything you, and others are saying, but we're not there yet. Sorry. - n On 8/1/07, Rick Cobb [EMAIL PROTECTED] wrote: I mostly sympathize with your sentiments. Incompatible changes should be documented. And design documents directions for AOLServer belong in public on the AOLServer wiki, not hidden. I.e., if when someone decided this was a good idea, there was an experimental features page for 4.5 that outlined the idea, that would at least form the basis for (a) docs, and (b) community discussion. If someone (like you or me) needed that backward compatibility, we could contribute it. At present, it still seems like the docs we see on the wiki are either generated from long-ago internal documentation, or not written at all. And as for roadmaps: Your search - roadmap - did not match any documents. However, I would like to point out that one of the real problems with the AOLServer community for a while was its perceived inability to pick up innovations. Discouraging change from inside AOL is only going to make the problem worse. We should be encouraging experimentation -- but with community feedback. -- ReC -Original Message- From: AOLserver Discussion [mailto: [EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, August 01, 2007 8:57 AM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command Technically everything in the configuration file is a Tcl command (ns_section, ns_param, etc.) so it's really not that much of a stretch. But I agree, it is different. Technically, the configuration file is a bucket of bits, but that's not a very useful observation. Had we ever actually gotten around to deploying AOLserver 4.5 internally here at AOL, I'm sure we would have at least done the work to provide some backwards compatibility wrappers for things like this, but as yet we haven't. Geezus. In this case, the code simply shouldn't've gone in. Or it should've been offered to the community in its incomplete form, with the offer to incorporate it in 4.5 if volunteers in the community completed it. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
To be honest and blunt, the community process in terms of AOLserver is rather broken, and has been for some time. The 4.5 work was done in-house here at AOL, and with limited community involvement. A lot of the experimental features were added to deal with specific scaling challenges we knew we would be facing here. When decided to go ahead and release 4.5, the thought was that we would eventually finish up any backwards compatibility work that needed to be done. We also assumed that if there were issues, people would simply hold off on upgrading, or work around the issues. Would documentation have helped - of course. Would a roadmap be useful - definitely. Agree with everything you, and others are saying, but we're not there yet. Sorry. - n On 8/1/07, Rick Cobb [EMAIL PROTECTED] wrote: I mostly sympathize with your sentiments. Incompatible changes should be documented. And design documents directions for AOLServer belong in public on the AOLServer wiki, not hidden. I.e., if when someone decided this was a good idea, there was an experimental features page for 4.5 that outlined the idea, that would at least form the basis for (a) docs, and (b) community discussion. If someone (like you or me) needed that backward compatibility, we could contribute it. At present, it still seems like the docs we see on the wiki are either generated from long-ago internal documentation, or not written at all. And as for roadmaps: Your search - roadmap - did not match any documents. However, I would like to point out that one of the real problems with the AOLServer community for a while was its perceived inability to pick up innovations. Discouraging change from inside AOL is only going to make the problem worse. We should be encouraging experimentation -- but with community feedback. -- ReC -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, August 01, 2007 8:57 AM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command Technically everything in the configuration file is a Tcl command (ns_section, ns_param, etc.) so it's really not that much of a stretch. But I agree, it is different. Technically, the configuration file is a bucket of bits, but that's not a very useful observation. Had we ever actually gotten around to deploying AOLserver 4.5 internally here at AOL, I'm sure we would have at least done the work to provide some backwards compatibility wrappers for things like this, but as yet we haven't. Geezus. In this case, the code simply shouldn't've gone in. Or it should've been offered to the community in its incomplete form, with the offer to incorporate it in 4.5 if volunteers in the community completed it. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Bingo - Michael Andrews just contributed such a script. - n On 8/1/07, Tom Jackson [EMAIL PROTECTED] wrote: I would just like to point out that 4.5 was released over one year ago, so the cat is already out of the bag. But more important: it seems like this is not really a problem. The old configuration can still be used. All we need is a script which reads the configuration data and runs the new commands. Is there a command reference anywhere, or could someone provide a simple example of use? If so, I can write a script and try to figure out where it should go so that it executes at the correct time. When is this time anyway? Apparently it is not during the sourcing of the config file. tom jackson On Wednesday 01 August 2007 09:33, Nathan Folkman wrote: As I said before, if you have issues with the changes made in 4.5, simply do not upgrade. There were a number of factors that led to our decision to release as we did, when we did. Unfortunately a lot of the backwards compatibility work never got completed. Sorry. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
} } } } - tom jackson On Wednesday 01 August 2007 10:50, Nathan Folkman wrote: Bingo - Michael Andrews just contributed such a script. - n On 8/1/07, Tom Jackson [EMAIL PROTECTED] wrote: I would just like to point out that 4.5 was released over one year ago, so the cat is already out of the bag. But more important: it seems like this is not really a problem. The old configuration can still be used. All we need is a script which reads the configuration data and runs the new commands. Is there a command reference anywhere, or could someone provide a simple example of use? If so, I can write a script and try to figure out where it should go so that it executes at the correct time. When is this time anyway? Apparently it is not during the sourcing of the config file. tom jackson On Wednesday 01 August 2007 09:33, Nathan Folkman wrote: As I said before, if you have issues with the changes made in 4.5, simply do not upgrade. There were a number of factors that led to our decision to release as we did, when we did. Unfortunately a lot of the backwards compatibility work never got completed. Sorry. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Probably would be that difficult, although honestly not a priority for anyone here. Have at it! ;) - n On 7/30/07, Jeff Rogers [EMAIL PROTECTED] wrote: Nathan Folkman wrote: Those parameters moved and are now controlled via the ns_pools Tcl command: This issue keeps coming up. How hard would it be to add in (for the next point release) compatibility code in init.tcl to parse the old-style parameters? -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
What version of AOLserver are you running? On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Hi Patform: suse 10.1 (2.6.16.21-0.25-default) I have configured my nsd config.tcl as below: ns_param maxconnections 100 ;# Max connections to put on queue ns_param maxdropped 0 ;# Shut down if dropping too many conns ns_param maxthreads 50;# Tune this to scale your server ns_param minthreads 20 ;# Tune this to scale your server ns_param threadtimeout 30 ;# Idle threads die at this rate ns_server threads command shows the min as zero, max as 10. Server Threads min0 max10 current1 idle0 stopping0 I'm not sure what i'm missing here. regards, shedi -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Those parameters moved and are now controlled via the ns_pools Tcl command: ns_pools: The ns_pools command enables configuration of one or more pools of connection processing threads. The pools allow certain requests to be handled by specific threads. This could, for example, ensure multiple long running requests don't block other short running requests. Pools are selected based on method/url pairs similar to the mappings managed by the ns_register_proc command. By default, all requests are handled by a single, unlimited, default pool. There is also an error pool as described below. Coupled with the new ns_limits command, pools can provide for sophisticated resource management. See also the new ns_limits command: ns_limit: The ns_limit command enables setting various resource limits for specified method/url combinations. These limits include such items as max concurrent connections, max file upload size, and timeouts waiting for connection processing. When limits are exceeded, connections are immediately dispatched to a dedicated error connection processing pool to generate a quick error response. By default all requests share the same default limits. Coupled with the new ns_pools command, URL-based limits can provide for sophisticated resource management. On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Its 4.5.0 On 7/26/07, Nathan Folkman [EMAIL PROTECTED] wrote: What version of AOLserver are you running? On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Hi Patform: suse 10.1 (2.6.16.21-0.25-default) I have configured my nsd config.tcl as below: ns_param maxconnections 100 ;# Max connections to put on queue ns_param maxdropped 0 ;# Shut down if dropping too many conns ns_param maxthreads 50;# Tune this to scale your server ns_param minthreads 20 ;# Tune this to scale your server ns_param threadtimeout 30 ;# Idle threads die at this rate ns_server threads command shows the min as zero, max as 10. Server Threads min0 max10 current1 idle0 stopping0 I'm not sure what i'm missing here. regards, shedi -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
You'd actually want to do it by adding the following to the end of your configuration file: ns_pools set procsmsgmgr -maxconns 100 -maxthreads 20 -minthreads 10 -timeout 10 ns_pools register procsmsgmgr server1 POST /proc/msgmgr You can then verify that everything worked via the AOLserver control port: server1:nscp 1 ns_pools list procsmsgmgr default error server1:nscp 2 ns_server threads procsmsgmgr {min 10} {max 20} {current 10} {idle 10} {stopping 0} server1:nscp 3 ns_pools get procsmsgmgr minthreads 10 maxthreads 20 idle 10 current 10 maxconns 100 queued 0 timeout 10 Hope that helps! - n On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Thanks Nathan for the info. I'm trying to configure a pool but ns_server threads procmsgmgr returns error on server log. [26/Jul/2007:18:06:07][4968.3074280352][-conn:0-] Error: Tcl exception: no such pool: procmsgmgr while executing ns_server threads procmsgmgr # ns_section ns/server/${servername}/pools ns_param procmsgmgr Message Manager Receiving Pool ns_section ns/server/${servername}/pool/procmsgmgr ns_param map {POST /proc/msgmgr} ns_param maxconnections 100 ns_param maxdropped 0 ns_param maxthreads 20 ns_param minthreads 10 ns_param threadtimeout60 Can you tell if my configuration is correct? regards, On 7/26/07, Nathan Folkman [EMAIL PROTECTED] wrote: Those parameters moved and are now controlled via the ns_pools Tcl command: ns_pools: The ns_pools command enables configuration of one or more pools of connection processing threads. The pools allow certain requests to be handled by specific threads. This could, for example, ensure multiple long running requests don't block other short running requests. Pools are selected based on method/url pairs similar to the mappings managed by the ns_register_proc command. By default, all requests are handled by a single, unlimited, default pool. There is also an error pool as described below. Coupled with the new ns_limits command, pools can provide for sophisticated resource management. See also the new ns_limits command: ns_limit: The ns_limit command enables setting various resource limits for specified method/url combinations. These limits include such items as max concurrent connections, max file upload size, and timeouts waiting for connection processing. When limits are exceeded, connections are immediately dispatched to a dedicated error connection processing pool to generate a quick error response. By default all requests share the same default limits. Coupled with the new ns_pools command, URL-based limits can provide for sophisticated resource management. On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Its 4.5.0 On 7/26/07, Nathan Folkman [EMAIL PROTECTED] wrote: What version of AOLserver are you running? On 7/26/07, Shedi Shedi [EMAIL PROTECTED] wrote: Hi Patform: suse 10.1 (2.6.16.21-0.25-default) I have configured my nsd config.tcl as below: ns_param maxconnections 100 ;# Max connections to put on queue ns_param maxdropped 0 ;# Shut down if dropping too many conns ns_param maxthreads 50;# Tune this to scale your server ns_param minthreads 20 ;# Tune this to scale your server ns_param threadtimeout 30 ;# Idle threads die at this rate ns_server threads command shows the min as zero, max as 10. Server Threads min0 max10 current1 idle0 stopping0 I'm not sure what i'm missing here. regards, shedi -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove
Re: [AOLSERVER] Example of nsproxy usage
From the AOLserver control port: server1:nscp 1 set handle [ns_proxy get myProxy] myProxy-proxy-0 server1:nscp 2 ns_proxy eval $handle [list exec uname -a] Darwin MacBook-Pro.local 8.10.1 Darwin Kernel Version 8.10.1: Wed May 23 16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386 i386 i386 Enjoy! - n On 7/13/07, JAMSHED QURESHI [EMAIL PROTECTED] wrote: Helo, Anyone like to give a taste of nsproxy usage for example an executable. cheers, -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Tcl exec hangs
You might also want to take a look at using an nsproxy. The nsproxy is essentially just a single-threaded tclsh that you can send commands to from your AOLserver. So what you could do is let the nsproxy perform the actually exec calls for you. Since the proxy is single-threaded, you shouldn't see the issue you are seeing when trying to do it from within AOLserver. Let me know if you need some example code. Hope that helps! - n On 7/10/07, Jeff Hobbs [EMAIL PROTECTED] wrote: Titi Alailima wrote: Periodically some of our AOLserver installations get into a mode where all calls to exec just hang, not really taking up any processor time but eating up a thread. Doesn't seem to be any memory problems coinciding, which I had originally suspected being a limiting factor on the forks. Besides, we usually get an error message when there's not enough memory to fork a child process. Haven't yet figured out anything in particular that causes this to start happening, but once it starts happening, it keeps going eating up more and more threads until we restart the nsd. Any ideas what the cause might be and what I can do to investigate further? We run AOLserver 4.0.10 on RHELAS 3, and CentOS 4, w/ Tcl 8.4.11. This is a known issue/conflict with pthreads and fork. Tcl does not yet have the pthread_at_fork functionality in place because that requires a careful marshaling of mutexes over the fork. You will find several core bug reports on this in the Tcl SF db area. It is possible to solve, its just that noone has invested the time yet. Jeff -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver 4.0.10 and Hoard 2.1.2d
You'll probably also want to make sure you're building Tcl such that it isn't using the Zippy threaded allocator code. Not sure if the Tcl folks ever fixed this, but it used to be that there was no easy way to build threaded Tcl without getting the Zippy allocator built as well. Using the Tcl threaded allocator may mask any benefits you'd see from using something like Hoard of TCMalloc since it maintains it's own per-thread memory pools and what not. - n On 5/24/07, Dossy Shiobara [EMAIL PROTECTED] wrote: On 2007.05.24, Agnieszka Kuka?owicz [EMAIL PROTECTED] wrote: Ns_TlsGet: invalid key: 0: should be between 1 and 100 This used to be a problem in older AOLserver 4.0, where the IP address you're using was missing reverse DNS ... but yours is failing in NsInitConf() ... are you sure you're loading the correct libnsd.so? I'd check your LD_LIBRARY_PATH and be extra-careful. -- Dossy -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Loading and initializing Tcl packages at startup
Along with this, I'd like to figure out what ns_ictl once actually does... ;-) I see that it adds items to a hash, but I never see any code acting on those hashed items. Perhaps this is the foundation for the init once functionality being sought? If so, we should complete the work in the head. - n On 5/7/07, Michael Andrews [EMAIL PROTECTED] wrote: There are times when you need to load Tcl packages @ startup. There are also times when you might want to have a one-time initialization of those packages before the server starts accepting connections. For example, you might have a package that registers a filter or a proc. The current implementation ns_ictl package require will re-require packages and re-eval namespaces on every interp clone. This is good for some reasons and should not be changed, but it does not give us a one-time initialization. I have added a tcl file to my AOLserver installation called packages.tcl that does the following: 1) Reads the nsd config to set library paths and load needed packages. 2) Looks for and executes the special command ::$ {packageName}::nsInit after ALL packages have been loaded. 3) Looks for and executes the special command ::$ {packageName}::nsPostInit after ALL nsInit procs have been executed. The config look like this: ns_section ns/server/server1/packages ns_param librarylist [list /Users/Michael/builds/aolserver/ packages] ns_param packagelist [list rpc TclCurl foo bar] This has really helped me pre-load packages for a core distribution and allowed me to use package deployments for AOLserver level code like ns_register_proc and ns_pool set becase I could start pools and register them before the server accepted connections. I want to get a sense of how folks need to load packages using AOLserver in hopes of adding this functionality to the AOLserver source. So I have some questions: 1) Is there a need to add package loading (like what I have done) to the core AOLserver distribution? 2) Is there a need to have a nsInit proc per package and when should that proc be executed: a) Right after right the package has been loaded - and before the other packages are loaded? (basically in line with the load list). b) After all the packages have been loaded (as I am doing in above) 3) Is there a need for a nsPostInit proc and when should it be executed: a) After all the nsInit procs have been executed but before the server accepts connections? (as I am doing above). b) After all the nsInit procs have been executed but AFTER the server accepts connections? 4) Should package load order be respected? 5) Are there other ns* procs that would be useful? 6) Other ideas on the config structure? I personally do not code in a way that needs load order, but others might, so my own opinion is that: Load order is easy to keep so why not; nsInit procs should be fired after ALL packages are loaded; and nsPostInit procs should be fired after ALL nsInits have been executed. This would give a nice way to initialize packages before the server accepts connections. I also think that any triggers that need to happen like ns_atshutdown can be registered in one of the init procs so we do not need to complicate things by have many special commands. Thoughts? M -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_db is not working
Make sure you are loading the nsdb.so module. Example: ns_section ns/server/server1/modules ns_param nsdb nsdb.so Hope that helps! - n On 5/3/07, Thorpe Mayes [EMAIL PROTECTED] wrote: I have recently installed aolserver 4.5. Everything seems to be working, except I get this error when I try to use the ns_db command: error: Tcl exception: invalid command name ns_db while executing ns_db gethandle $pool invoked from within set db [ns_db gethandle $pool] ns_info works fine. I know I am missing the obvious. Thorpe -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] (Resolved) Re: nsthreadtest and segmentation fault
That's just a hunch about that being the cause, but it is true, AOLserver has not been thoroughly reviewed or tested as to be truly 64-bit clean, so you may see issues. - n On 4/17/07, aT [EMAIL PROTECTED] wrote: I found out (thanks to Nathan) that nsthreadtest is crashing on my servers because i compiled aolserver with --enable-64bit . aolserver 64 bit code is still not thoroughly tested no wonder its crashing on my 64 bit server . On the same machine running 32bit compiled versions works fine . so I will avoid compiling aolserver with --enable-64bit for now. However i must mention this here that i have run benchmarks with and without 64bit on aolserver and found 15% to 20% performance enhancement . i might have to do that again :) Thanks On 4/15/07, aT [EMAIL PROTECTED] wrote: Running nsthreadtest always produces segmentation fault on my servers. I have tried with Aolserver 4.0.10 and aolserver 4.5.0 on , amd x86_64 running 2.6.9-42.ELsmp AND 2.4.21-27.ELsmp doing gdb on core file gives this aolserver was compiled with --enable-threads Loaded symbols for /lib64/ld-linux-x86-64.so.2Core was generated by `./bin/nsthreadtest'. Program terminated with signal 11, Segmentation fault. #0 0x002a95a58bd8 in pthread_join () from /lib64/tls/libpthread.so.0 -- Syed Atif Ali Desk: 971 4 3911914 -- Syed Atif Ali Desk: 971 4 3911914 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_zlib compress
I'd have to look again to confirm whether or not ns_return wasn't binary safe, but I wouldn't be surprised. I recently patched ns_http to correctly handle the sending and receiving of binary data. I'll try and take a look at as ns_return next. Would you mind adding this as a bug? Thanks! - n On 4/19/07, Rusty Brooks [EMAIL PROTECTED] wrote: I don't believe you can use ns_return with binary data. By the way, are there any plans to change that? Or make a ns_return_binary or something? I have a lot of code that writes to a file and then does ns_returnfile Rusty John Buckman wrote: How do you enable gzip page compression on ns_register_proc pages? The adp gzip compression occurs in C code, in Ns_ConnFlush(), which checks for Accept-Encoding: gzip Since a ns_register_proc uses ns_return, like this: ns_return $conn 200 text/html $html I believe it's bypassing this gzip code, and always returning html. I wrote a ns_return_gzipped() proc (see below), but it doesn't seem to work, so I was wondering if anyone's done this already, and how. FYI I switched BookMooch to use adp gzip compression, and the results are impressive. 200k html files go to 35k, greatly speeding up the user experience. -john proc ns_return_gzipped {conn html} { if {[string first {gzip} [ns_set get [ns_conn headers] {Accept-Encoding}]] != -1} { set zl [ns_zlib gzip $html] ns_set put [ns_conn outputheaders] Content-Encoding gzip ns_return $conn 200 text/html $zl } else { ns_return $conn 200 text/html $html } } -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] profiling an aolserver site
I added server-side logging of ADP page execution a while back. Maybe that would help you? ns_section ns/server/server1/module/nslog ns_param logreqtime true [MacBook-Pro:~] nfolkman% tail -f /usr/local/aolserver/servers/server1/modules/nslog/access.log 192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] GET /flash/ HTTP/1.1 200 191 Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 0.106342 192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] GET /favicon.ico HTTP/1.1 404 534 Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 0.000361 On 4/11/07, John Buckman [EMAIL PROTECTED] wrote: I'm wondering if anyone on this list has written code to profile a web site running under Aolserver. By this, I mean, timing the start/stop time of every page, logging it, and then running a bit of analysis to find out what pages are the slowest running and which pages are the most commonly loaded, then multiplying the two (ie execution time x requests per day=total machine load per page per day) In traditional systems programming, profiling is a common tool used to determine what code should be optimized. I'd like to do the same inside aolserver. One efficient alternative I was thinking about would be to patch ns_log to include both the start request time, and the time the page was returned, in the log. That could be done if ns_log is called after the page is rendered, and I don't know if that's the case. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] profiling an aolserver site
It *should* handle all connection requests. The timing happens in the LogTrace() funtion in nslog/nslog.c. That trace generally fires last to do logging to the access log, and the request execution time is computed by comparing the time that LogTrace() was called to the time returned by Ns_ConnStartTime(). Probably is a better way to do this, but this will at least give you a consistent baseline to compare page execution times. I don't have any analysis scripts, but would suggest looking at R. - n On 4/11/07, John Buckman [EMAIL PROTECTED] wrote: I added server-side logging of ADP page execution a while back. Maybe that would help you? ns_section ns/server/server1/module/nslog ns_param logreqtime true Yes, that's great. Will it also log ns_register_proc pages? Have you written anything to crunch the numbers? If not, no problem, I can do that. -john [MacBook-Pro:~] nfolkman% tail -f /usr/local/aolserver/servers/server1/modules/nslog/access.log 192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] GET /flash/ HTTP/1.1 200 191 Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv: 1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 0.106342 192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] GET /favicon.ico HTTP/1.1 404 534 Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv: 1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 0.000361 On 4/11/07, John Buckman [EMAIL PROTECTED] wrote: I'm wondering if anyone on this list has written code to profile a web site running under Aolserver. By this, I mean, timing the start/stop time of every page, logging it, and then running a bit of analysis to find out what pages are the slowest running and which pages are the most commonly loaded, then multiplying the two (ie execution time x requests per day=total machine load per page per day) In traditional systems programming, profiling is a common tool used to determine what code should be optimized. I'd like to do the same inside aolserver. One efficient alternative I was thinking about would be to patch ns_log to include both the start request time, and the time the page was returned, in the log. That could be done if ns_log is called after the page is rendered, and I don't know if that's the case. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] couldn't load file libnszlib.so: /software/aol/aol45/lib/libnszlib.so: undefined symbol: compressBound
Make sure the path to the dependent libraries is in your LD_LIBRARY_PATH. - n On 4/10/07, aT [EMAIL PROTECTED] wrote: I have done that , I tried both ways of loading nszlib , module and library way . Even then i get the same error. Any Hint about the Error ? Dossy Shiobara wrote: On 2007.04.10, aT [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Do i need load it from configuration file where other modules are loaded: ns_section ns/server/test/modules or i can just do load libnszlib.so in the application ? You have to load it in the modules section in the config. -- Dossy -- Syed Atif Ali D. +971 4 3911914 F. +971 4 3911915 ___ The world is coming to an end! Repent and return those library books! -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AolServer and core dumps
You might want to take a look at the Tcl Core Dumper package Dossy and I did a while back. http://aolserver.cvs.sourceforge.net/aolserver/tcl-coredumper/ - n On 3/28/07, Agnieszka Kukałowicz [EMAIL PROTECTED] wrote: Hi, I have question on how to force Aolserver to dump core? My Aolserver crashes in some circumstances causing segmentation fault. I would like to debug the problem, but I can't see any core file. It looks like everything is set right. I made 'ulimit -c unlimited' to have unlimited core file size and my permissions where I dump a core look ok. What might be the reason it does not work? Cheers Agnieszka Kukalowicz -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED]
Re: [AOLSERVER] /tmp/files disappearing after request is processed
Take a look at: ns_getform Looks like it's getting removed here: form.tcl: 160 ns_atclose ns_unlink -nocomplain $tmpfile Can you update your code to copy the returned /tmp file to your own directory? Probably safer that way anyways, since you never know when /tmp will be cleaned up. - n On 3/1/07, Pedro Liska [EMAIL PROTECTED] wrote: Hi! I'm moving my web server from AOLserver 3.3 to AOLserver 4.5. With my old AOLserver 3.3 configuration, the /tmp/files posted by a multipart/form-data form would remain after the request was processed. This is not the case in my AOLserver 4.5 test configuration. Do you guys know how to make AOLserver 4.5 not delete the /tmp/files after the request is processed? My Tcl code depends on the files being there in several cases. Thanks! Pedro -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] tdom leak fix
Don't you open yourself up for a race condition if that code is in a scheduled proc? On 2/28/07, Jeff Rogers [EMAIL PROTECTED] wrote: Nick Miller wrote: Heya, I have added: foreach lostdom [info commands domDoc0*] { $lostdom delete } As you say though, best to make sure you catch them when your dealing with them and explicitly delete them. Thanks Nick Having run into this same problem before I did something similar, setting up the above as a scheduled proc. Looking at my code I also included eval unset [info vars ::dom::_node*Attributes] although I don't recall if this was for tdom or tclxml. -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] tdom leak fix
I believe ns_ictl is what you would want - ns_ictl oncleanup I believe. Another option is to run all request through a configured start page and to put the cleanup code in that .adp page. Nice thing about that approach is that you can make changes without having to restart the server. Configuration: ns_section ns/server/server1/adp ns_param startpage PATH/start.adp start.adp: % _ns_adp_include [ns_url2file [ns_conn url]] # # tDOM cleanup code goes here. # % On 2/28/07, Tom Jackson [EMAIL PROTECTED] wrote: Yes, wouldn't it be nice if ns_atclose worked in scheduled procs? In a conn you can use ns_atclose, but it has been removed from scheduled procs. Is there no way to run a trace on a scheduled proc? I have looked at ns_ictl, but it looks like the scheduling is per module, so you can't just activate it when you want like ns_atclose. Not having to catch code is a big benefit, being able to use the same code in all threads would be very nice. tom jackson On Wednesday 28 February 2007 11:41, Nathan Folkman wrote: Don't you open yourself up for a race condition if that code is in a scheduled proc? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] tdom leak fix
Cool. BTW, I wouldn't call this a leak, and would instead call it misuse of tDOM. ;-) On 2/28/07, Jeff Rogers [EMAIL PROTECTED] wrote: Nathan Folkman wrote: Don't you open yourself up for a race condition if that code is in a scheduled proc? If it was an aolserver scheduled proc then yes, but in my application it was running in an after rescheduling loop within tclhttpd so there was no risk of pre-emption; but the memory (dom object) issue was pretty much the same. -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Memory Leaks of tDOM
Well, there's the ns_info pools Tcl command. Also look at aolserver/modules/tcl/stats.tcl for example usage. On 2/27/07, Nick Miller [EMAIL PROTECTED] wrote: Hey Nathan, I have run into an issue with yea oldie memory leak problem. Was chatting to Dave about this earlier on and am trying to find something to quickly get usable memory usage information back on AOLServer. I am not sufficiently skilled in the ways of memory use by Tcl to be able to quickly put something together and wondered if you knew of anything that might be a useful tool for this? Mr Schroeder mentioned that once upon a time we had some compiled c code that was used to debug memory leak problems - do you know anything about this? I hope all is well with you. nick -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_proxy
One thing you can try is to have your proxies start up when the server starts. You can warm them up by sending down a command at server startup. On 2/25/07, Malte Sussdorff [EMAIL PROTECTED] wrote: Due to the fact that we are experiencing a couple of problems with exec in AOLserver 4.5 with TCL 8.4.14 due to out of memory, I wanted to run the TCL exec command with nsproxy. Sadly, ns_proxy get test fails as well with not enough memory. Now, I am wondering, what else can be done. The memory is at 4GB at the moment. There might be an issue with the OpenACS site_node cache, but this is a nsv_array, so it should not be replicated in the exec process, or am I wrong? Any hint is highly welcome, maybe I need to setup ns_proxy Pools just like database pools ? Thanks Malte -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] mutex try lock?
Just curious, but what's the use case where you would use something like this? - n On 2/22/07, Tom Jackson [EMAIL PROTECTED] wrote: I have some example code of condition wait: http://rmadilo.com/m2/servers/rmadilo/modules/tcl/twt/packages/db/tcl/datasource-procs.tcl The procs lock, unlock, wait, popPool, etc. Wrap the ideas up pretty easily, but the question is what happens when you code times out waiting for a resource? tom jackson On Thursday 22 February 2007 07:09, John Buckman wrote: I'm looking to have mutexes with timeouts, and I see support in the C code for this but none carried over to Tcl. In the C code, there's a Ns_MutexTryLock() function, but no tcl function for calling it. Ns_MutexLock calls Ns_MutexTryLock() and there appears to be timeout support: if (!NsLockTry(mutexPtr-lock)) { return NS_TIMEOUT; } but I don't see any way of setting the mutex timeout seconds. It's trivially simple to modify NsTclMutexObjCmd (in tclthread.c) to support ns_mutex try and I'm wondering if there's a reason this hasn't been done. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TDOM and OSX
You'll probably want to also compile Tcl yourself. Not sure the version included with OS X is compiled with support for threads, but you'd have to check... Tcl: ./configure --enable-threads ... If you don't compile Tcl with support for threads, you won't get the fast Zippy allocator. - n On 2/21/07, Michael Andrews [EMAIL PROTECTED] wrote: use your own paths for tcl and aolserver in the configure command of course... M On Feb 21, 2007, at 1:22 PM, Michael Andrews wrote: I had this same issue. You need to get and compile tdom from CVS. Do not get it from a mirror on http://www.tdom.org/ Do this: cvs -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs login cvs -z3 -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs co tdom ../configure --enable-symbols --with-tcl=/usr/local/aolserver/lib --prefix=/usr/local/aolserver --with-aolserver=/usr/local/aolserver That'll fix your issue. On Feb 21, 2007, at 12:48 PM, John Divney wrote: Hi, I am having trouble w/ tdom on OS X. I am able to run make/make install w/out issue. This copies the libraries to the aolserver directory, but when I start AOLServer I get the following error: [21/Feb/2007:12:44:52][382.2684407744][-main-] Error: tcl: source /usr/local/aolserver/modules/tcl/init.tcl failed: couldn't find procedure Tdom_Init NONE couldn't find procedure Tdom_Init while executing ns_eval package require tdom (file /usr/local/aolserver/modules/tcl/init.tcl line 64) invoked from within source $file I am calling package require from aolserver/modules/tcl/init.tcl The libraries are in /aolserver/lib/tdom0.8.0 When I grep for Tdom_Init is is only in the .a file, not in the .so or .dylib file, so I think I am doing something wrong w/ my CONFIG file. Here is how I am calling that. ../configure \ --mandir=/usr/local/share/man \ --enable-threads \ --prefix=/usr/local/aolserver \ --libdir=/System/Library/Tcl \ --with-tcl=/System/Library/Frameworks/Tcl.framework \ --with-tclinclude=/System/Library/Frameworks/Tcl.framework/Headers I have tried a ton of alternatives, read through posts on openacs site, and some others, but nothing seems to work. Any help is appreciated ! Thanks! div -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TDOM and OSX
1. Check and see if ns_info pools returns any data. 2. Here's one option: http://developer.apple.com/macosx/launchd.html - n On 2/21/07, John Divney [EMAIL PROTECTED] wrote: This did the trick. Thanks for all the suggestions. A couple more questions: 1) How do I know zippy is working. I did compile TCL myself. Is there something I can try to verify ? 2) What is equivalent of init.tab on OS X ? Thanks! div Michael Andrews wrote: I had this same issue. You need to get and compile tdom from CVS. Do not get it from a mirror on http://www.tdom.org/ Do this: cvs -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs login cvs -z3 -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs co tdom ../configure --enable-symbols --with-tcl=/usr/local/aolserver/lib --prefix=/usr/local/aolserver --with-aolserver=/usr/local/aolserver That'll fix your issue. On Feb 21, 2007, at 12:48 PM, John Divney wrote: Hi, I am having trouble w/ tdom on OS X. I am able to run make/make install w/out issue. This copies the libraries to the aolserver directory, but when I start AOLServer I get the following error: [21/Feb/2007:12:44:52][382.2684407744][-main-] Error: tcl: source /usr/local/aolserver/modules/tcl/init.tcl failed: couldn't find procedure Tdom_Init NONE couldn't find procedure Tdom_Init while executing ns_eval package require tdom (file /usr/local/aolserver/modules/tcl/init.tcl line 64) invoked from within source $file I am calling package require from aolserver/modules/tcl/init.tcl The libraries are in /aolserver/lib/tdom0.8.0 When I grep for Tdom_Init is is only in the .a file, not in the .so or .dylib file, so I think I am doing something wrong w/ my CONFIG file. Here is how I am calling that. ../configure \ --mandir=/usr/local/share/man \ --enable-threads \ --prefix=/usr/local/aolserver \ --libdir=/System/Library/Tcl \ --with-tcl=/System/Library/Frameworks/Tcl.framework \ --with-tclinclude=/System/Library/Frameworks/Tcl.framework/Headers I have tried a ton of alternatives, read through posts on openacs site, and some others, but nothing seems to work. Any help is appreciated ! Thanks! div -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] make install problem
The AOLserver 4.5 make process is slightly different then previous versions: % cd aolserver % ./configure --enable-symbols --prefix=/usr/local/aolserver --with-tcl=/usr/local/aolserver/lib % /usr/local/aolserver/bin/tclsh8.4 nsconfig.tcl -debug -install /usr/local/aolserver % make % make install On 2/6/07, Arturo Mardones [EMAIL PROTECTED] wrote: Hi! I'm installing aolserver on mandriva 2007, and i have this packages: libtcl8.4-devel-8.4.13-1mdk libtcl8.4-8.4.13-1mdk tcl-8.4.13-1mdk and daemontools but this output have when try install [EMAIL PROTECTED] ~]# cd /usr/local/src/aolserver-4.5.0/ [EMAIL PROTECTED] aolserver-4.5.0]# ./configure --enable-threads checking for Tcl configuration... found /usr/lib/tclConfig.sh checking for existence of /usr/lib/tclConfig.sh... loading checking for build with symbols... no checking for gcc... gcc -pipe checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc -pipe accepts -g... yes checking for gcc -pipe option to accept ANSI C... none needed checking how to run the C preprocessor... gcc -pipe -E checking for egrep... grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for inttypes.h... (cached) yes checking for timegm... yes checking for fork1... no checking for drand48... yes checking for random... yes checking for _NSGetEnviron... no checking for poll... yes checking dl.h usability... no checking dl.h presence... no checking for dl.h... no checking need for dup high... no checking for shared libgcc... yes checking for getaddrinfo in -lsocket... no checking for getnameinfo in -lsocket... no checking for getaddrinfo... yes checking for getnameinfo... yes configure: creating ./config.status config.status: creating include/ns.mak [EMAIL PROTECTED] aolserver-4.5.0]# make install ./util/nsmakeall.tcl install nsthread nsd nstclsh ./util/nsmakeall.tcl: line 48: syntax error near unexpected token `}' ./util/nsmakeall.tcl: line 48: `}' make: *** [install-bins] Error 2 Thanks for any help!! -- http://animaldelared.blogspot.com -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_ctl
We're behind on the 4.5 documentation... Look for things to pick up with 4.5 development soon. ;-) On 1/12/07, Gustaf Neumann [EMAIL PROTECTED] wrote: Dear all. Is it intentional that ns_adp_ctl is not mentioned on http://panoptic.com/wiki/aolserver/Tcl_API ? One gets no friendly feedback, no adp output context when one tries to ns_adp_parse a file larger than 1024000 bytes. It took me a while to find the reason. the 4.5 function ns_adp_ctl bufsize can be easily increased for such cases (default 1024000). other options of ns:adp_ctl are: bufsize, channel, autoabort, detailerror, displayerror, expire, gzip, nocache, safe, singlescript, stricterror, trace, trimspace, best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver 4.0.10 and Hoard
Are you trying to preload Hoard with a version of Tcl that was built with the threaded allocator? If so, you're not going to realize much in the way of performance or memory savings since higher-level allocations will still ultimately be going through Tcl's threaded (bucket) allocator. On 1/2/07, Agnieszka Kukałowicz [EMAIL PROTECTED] wrote: Hi, I have problem with running AOLserver 4.0.10 with Hoard on my production server. After putting in my run script the line below: export LD_PRELOAD=/usr/lib/libdl.so:/usr/local/lib/libhoard.so:/usr/lib/libpth rea.so and restarting the server I get: Ns_TlsGet: invalid key: 0: should be between 1 and 100 When I tested Hoard on my developer server everything was ok. What am I doing wrong? Cheers Agnieszka Kukałowicz -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED]
Re: [AOLSERVER] aolserver bug
Saw. Cool. Thanks! On 12/15/06, Dossy Shiobara [EMAIL PROTECTED] wrote: On 2006.12.14, Gustaf Neumann [EMAIL PROTECTED] wrote: I am following up a posting from January, describing a problem which lead to frequent hangs on openacs.org. [...] i would recommend to add this or a similar fix to the code base of aolserver 4.5, since the reported problem persists as well in the new driver of 4.5 I've gone and accepted the patch for both HEAD and aolserver_v40_bp. Refer to SF BUG #1615787: http://aolserver.com/sf/bug/1615787 Thanks! -- Dossy -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver bug
I'll take a look at this over the next couple of weeks when I'm on vacation and see if I can get it applied. Thanks for finding and fixing! :-) On 12/14/06, dhogaza@pacifier.com dhogaza@pacifier.com wrote: i would recommend to add this or a similar fix to the code base of aolserver 4.5, since the reported problem persists as well in the new driver of 4.5 Let me second Gustaf on this. Our site (openacs.org) was locking up every few hours for some weeks before we applied this patch. After applying it, we've gone two full days without lock-up. This bug is real, and this or another fix really need to get into the code. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] tcmalloc
There actually is a fair amount of documentation, just most of it is in the source file. Take a look at: zippy2/README zippy2/tclThreadAlloc.c Zippy2 allows you to override the Tcl threaded allocator functions, so you don't need to worry about recompiling Tcl with USE_THREAD_ALLOC=0 in this case. To run, you'd do something like: env -i LD_PRELOAD=libhoard.so:libzippy.so TCL_ALLOC_DISABLE=1 bin/nsd -ft nsd.tcl The Zippy2 allocator is a complete work in progress, and has never been deployed in production. It might be better to look at the performance of threaded Tcl without the thread allocator (USE_THREAD_ALLOC=0) and hoard. Hope that helps! On 12/14/06, Agnieszka Kukałowicz [EMAIL PROTECTED] wrote: For sure… Followed the instructions for Hoard for example, I ony set the LD_PRELOAD variable. And there is also no instructions for zippy2 to recompile TCL without DUSE_THREAD_ALLOC. So my question is if I want to use Hoard/Zippy2/Tcmalloc do I need to built TCL without the zippy memory allocator as you wrote? And the second thing… Maybe, It would be a good idea to put these instruction in README file in Zippy2 code? Agnieszka -Original Message- *From:* AOLserver Discussion [mailto:[EMAIL PROTECTED] *On Behalf Of *Nathan Folkman *Sent:* Wednesday, December 13, 2006 8:07 PM *To:* AOLSERVER@LISTSERV.AOL.COM *Subject:* Re: [AOLSERVER] tcmalloc Exactly right. Thanks for jogging my memory! ;-) Here's a snippet from one of our custom build scripts which might help: @echo ** MODIFYING TCL TO BUILD WITHOUT THE ZIPPY MEMORY ALLOCATOR ** cd $(ROOT_DIR)/tcl/unix \ cat Makefile | sed -e 's/-DUSE_THREAD_ALLOC=1//' Makefile.new \ cat tclConfig.sh | sed -e 's/-DUSE_THREAD_ALLOC=1//' tclConfig.sh.new \ cp Makefile.new Makefile \ cp tclConfig.sh.new tclConfig.sh On 12/13/06, *Jeff Rogers* [EMAIL PROTECTED] wrote: Nathan Folkman wrote: What you need to do is build a version of Tcl that has support for threads, but does not use the Zippy allocator. I seem to remember there being issues with core Tcl that did not allow you to do this easily, but you'd have to double check. Maybe try something like --enable-threads --disable-thread-alloc. If I recall from when I was poking at this, there is no configure flag to disable the threaded allocator; instead you need to edit the generated Makefile after you run configure and remove -DUSE_THREAD_ALLOC=1 from AC_FLAGS. And of course, 'make clean' before remaking or nothing will change. -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED]
Re: [AOLSERVER] tcmalloc
Always know your audience... ;-) Let me start over as to make sure I'm not leading anyone astray... Tcl can be built in one of two ways: single-threaded, which is the default, or threaded, which requires you to configure things with the -enable-threads flag. When you build Tcl with support for threads, the threaded allocator, aka Zippy, is enabled. At a high level, the Zippy allocator is a x2 allocator which is optimized around lock avoidance that comes at the cost of relatively high memory overhead. Ultimately the Zippy allocator uses malloc() and free() for block allocations, and mmap() for large allocations, but the allocated blocks flow back and forth between a shared pool and per-thread pools. There is also no automatic garbage collection. The allocator basically caches memory in both shared and per-thread pools as to avoid having to go to the system to get memory. If you were to try and pre-load an alternate memory allocator for threaded Tcl whichused the Zippy allocator by default, you wouldn't see much difference, since the higher memory usage is a side affect of how the Zippy allocator is functions - x2 bucket allocations, and how that allocated memory is passed between the shared and per-thread pools. What you need to do is build a version of Tcl that has support for threads, but does not use the Zippy allocator. I seem to remember there being issues with core Tcl that did not allow you to do this easily, but you'd have to double check. Maybe try something like --enable-threads --disable-thread-alloc. The newer zippy2 code exposes an environment variable that allows you to turn off the threaded allocator at load time. Example: env -i TCL_ALLOC_DISABLE=1 LD_PRELOAD=libhoard.so bin/nsd ... Setting the TCL_ALLOC_DISABLE environment variable will send all allocations directly to malloc() and free() which can then be overridden by the LD_PRELOAD of an alternate memory allocator. Hope that helps? On 12/13/06, aT [EMAIL PROTECTED] wrote: I understand developers might be able to understand what you mentioned , being a sysadmin it is a bit hard for me to understand LD_PRELOAD and compiling tcl not to use threaded allocator . i would really appreciate if the above mentioned basically paragraph be explained more . * Basically what you need to do is compile Tcl so that it doesn't use the threaded allocator, and instead uses regular malloc() and free(). * I tried in tcl source ./configure --help to find out how can i do what u mentioned , but could not find any help how to , not use, threaded allocator and instead use malloc() and free() . * Then compile tcmalloc, * If i compile tcmalloc is this a standalone procedure or do i need to mention aolserver location in anyway . * and finally LD_PRELOAD tcmalloc at AOLserver startup time. This will override malloc() and free(). * How do i do that ? Nathan Folkman wrote: Basically what you need to do is compile Tcl so that it doesn't use the threaded allocator, and instead uses regular malloc() and free(). Then compile tcmalloc, and finally LD_PRELOAD tcmalloc at AOLserver startup time. This will override malloc() and free(). Hope that helps! - n On 12/11/06, aT [EMAIL PROTECTED] wrote: Nathan Folkman wrote: Anyone tried Google's tcmalloc with AOLserver instead of using Tcl's threaded allocator? Looks like folks are seeing some good things with it. http://dammit.lt/2006/12/06/google-perftools-tcmalloc-squid/ -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. I would like to try it , but have no clue how to compile this against aolserver. can anyone post a small howto on compiling it with aolserver ? -- Syed Atif Ali D. +971 4 3911914 F. +971 4 3911915 ___ The sum of the intelligence on the planet is a constant; the population is growing. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Syed Atif Ali D. +971 4 3911914 F. +971 4 3911915 ___ Walk softly and carry a megawatt laser. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank
Re: [AOLSERVER] tcmalloc
Exactly right. Thanks for jogging my memory! ;-) Here's a snippet from one of our custom build scripts which might help: @echo ** MODIFYING TCL TO BUILD WITHOUT THE ZIPPY MEMORY ALLOCATOR ** cd $(ROOT_DIR)/tcl/unix \ cat Makefile | sed -e 's/-DUSE_THREAD_ALLOC=1//' Makefile.new \ cat tclConfig.sh | sed -e 's/-DUSE_THREAD_ALLOC=1//' tclConfig.sh.new \ cp Makefile.new Makefile \ cp tclConfig.sh.new tclConfig.sh On 12/13/06, Jeff Rogers [EMAIL PROTECTED] wrote: Nathan Folkman wrote: What you need to do is build a version of Tcl that has support for threads, but does not use the Zippy allocator. I seem to remember there being issues with core Tcl that did not allow you to do this easily, but you'd have to double check. Maybe try something like --enable-threads --disable-thread-alloc. If I recall from when I was poking at this, there is no configure flag to disable the threaded allocator; instead you need to edit the generated Makefile after you run configure and remove -DUSE_THREAD_ALLOC=1 from AC_FLAGS. And of course, 'make clean' before remaking or nothing will change. -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] AOLserver Memory Allocator Performance
Couldn't resist looking at this... :-) Linux localhost.localdomain 2.6.9-34.EL #1 Fri Feb 24 16:44:51 EST 2006 i686 i686 i386 GNU/Linux AOLserver 4.5 + Tcl 8.4.14 (USE_THREAD_ALLOC=0) % bin/nsthreadtest starting 10 malloc threads...waitingdone: 0 seconds, 159637 usec starting 10 ns_malloc threads...waitingdone: 0 seconds, 102511 usec AOLserver 4.5 + Tcl 8.4.14 (USE_THREAD_ALLOC=1) % bin/nsthreadtest starting 10 malloc threads...waitingdone: 0 seconds, 104061 usec starting 10 ns_malloc threads...waitingdone: 0 seconds, 109929 usec AOLserver 4.5 + Tcl 8.4.14 (USE_THREAD_ALLOC=0) + Google Perf Tools (tcmalloc) 0.8.1 % env -i LD_PRELOAD=lib/libtcmalloc.so bin/nsthreadtest starting 10 malloc threads...waitingdone: 0 seconds, 91301 usec starting 10 ns_malloc threads...waitingdone: 0 seconds, 107320 usec Clearly we need a better set of tests... Anyone got any ideas? Seems like the two things we'd want to compare are performance and overall memory usage. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] tcmalloc
Basically what you need to do is compile Tcl so that it doesn't use the threaded allocator, and instead uses regular malloc() and free(). Then compile tcmalloc, and finally LD_PRELOAD tcmalloc at AOLserver startup time. This will override malloc() and free(). Hope that helps! - n On 12/11/06, aT [EMAIL PROTECTED] wrote: Nathan Folkman wrote: Anyone tried Google's tcmalloc with AOLserver instead of using Tcl's threaded allocator? Looks like folks are seeing some good things with it. http://dammit.lt/2006/12/06/google-perftools-tcmalloc-squid/ -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. I would like to try it , but have no clue how to compile this against aolserver. can anyone post a small howto on compiling it with aolserver ? -- Syed Atif Ali D. +971 4 3911914 F. +971 4 3911915 ___ The sum of the intelligence on the planet is a constant; the population is growing. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] zippy2 Build Problem for AOLserver 4.0.x
Glad it worked. :-) The Tcl threaded allocator, which is derived from the original AOLserver zippy allocator, is optimized solely for lock avoidance, and this comes at the cost of memory foot print. In fact, there's no garbage collection for smaller allocations. Instead, memory moves between the shared and per-thread pools. There are a couple of additional things you could try which may also help. Make sure you configure your connection threads such that they eventually exit. I believe by default they stick around forever and get re-used. This can be a good thing, but depending on the max number of connection threads you have configured, you may tying up a bunch of memory in the per-thread pool. When connection threads exit, the memory from the per-thread pool is returned to the shared pool. Another technique which has worked well is to move memory intensive tasks into the nsproxy. The nsproxy is a single-threaded tclsh which uses system malloc() and free(), and which will also return memory to the system on exit. We played around with various options in the new zippy2 allocator, but never saw any major improvements, at least for the types of systems we ran at AOL. If you have time, you may want to try out Google's tcmalloc, as its being getting a lot of positive feedback, and is supposed to be optimized for threaded applications. On 12/11/06, Agnieszka Kukałowicz [EMAIL PROTECTED] wrote: Hi, Try the following as a replacement for the current Makefile: CLEAN = clean-it INSTALL = install-it LIB = zippy2 LIBOBJS = tclThreadAlloc.o CFLAGS += -I$(TCL)/generic ... It works. Thanks a lot. Also curious as to what motivated you to look into this in the first place. Are you experiencing memory issues? Are you experiencing memory issues? Yes, My production server uses a lot of memory (almost 2GB after one day running). First, I thought it was because of some kind of bad code in our application. But I made a lot of tests and statistics and I did notice nothing. The application is very stable, there is no leak of memory. My conclusion was it was due to memory alloction in AOLserver and articles in the Internet confirmed that. Now I'm testing AOLserver with Hoard and I need to say it looks very nice. The memory usage decreased about 20% - 40% depending on server parameters. I would also like to see how zippy2 works and compare it with hoard. I will let you know about results of my experiment. Cheers, Agnieszka -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Nathan Folkman [EMAIL PROTECTED]
[AOLSERVER] tcmalloc
Anyone tried Google's tcmalloc with AOLserver instead of using Tcl's threaded allocator? Looks like folks are seeing some good things with it. http://dammit.lt/2006/12/06/google-perftools-tcmalloc-squid/ -- Nathan Folkman [EMAIL PROTECTED] -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] test
test -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] zippy2 problem
[EMAIL PROTECTED] wrote: Hi, If memory serves, the zippy2 work was done against AOLserver 4.5, which uses slightly different semantics for building. I think you should be able to simply change that line to be: Makefile:41: [.]/aolserver/include/Makefile.module Unfortunately it doesn't work. While building, I get errors: tclThreadAlloc.c:19:20: tclInt.h: No such file or directory tclThreadAlloc.c:20:21: tclPort.h: No such file or directory I guess it is because of a problem to find TCL include directory. But I don't know what to change in order to make it work. Please, help. Cheers Agnieszka Kukalowicz -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Are you trying to build this for AOLserver 4.0.x? - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] zippy2 Build Problem for AOLserver 4.0.x
Try the following as a replacement for the current Makefile: CLEAN = clean-it INSTALL = install-it LIB = zippy2 LIBOBJS = tclThreadAlloc.o CFLAGS += -I$(TCL)/generic include $(AOLSERVER)/include/Makefile.module $(LIB): $(OBJS) $(LDLIB) $(LDFLAGS) -o $(LIB) $(LIBOBJS) $(CLEAN): $(RM) $(LIBOBJS) $(LIB) $(INSTALL): $(BUILD) $(INST) -d $(INSTLIB) -e $(LIB) You'll need to set TCL to point to the Tcl source directory. One other option, and something you'd have to do if on a Mac, is to simply replace the tclThreadAlloc.c file in your Tcl source tree with the one provided here, rebuild Tcl, then build the alloccmd stuff (make -f Makefile.cmd). Also curious as to what motivated you to look into this in the first place. Are you experiencing memory issues? Hope that helps! - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] zippy2 problem
[EMAIL PROTECTED] wrote: Hi, I have problem compiling zippy2 allocator. I hoped to make some tests to compare Zippy2 with Hoard allocator, but when I tried to build zippy2 I get errors: Makefile:41: […]/aolserver/include/ns.mak: No such file or directory I think it is due to missing ns.mak file in version 4.0.10 I have. So, my question is what I should change in Makefile for to build zippy2 without errors. I use AOLserver in 4.0.10 in my production environment and so far I don’t want to upgrade it to 4.5 version. Agnieszka Kukalowicz -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. If memory serves, the zippy2 work was done against AOLserver 4.5, which uses slightly different semantics for building. I think you should be able to simply change that line to be: Makefile:41: […]/aolserver/include/Makefile.module Below are a few other items which are different between 4.5 and previous versions. Hope that helps. Let me know if that doesn't work. Also let me know if you need any help getting things set up to test with hoard. - n Makefile Incompatibilities Previous server versions: MOD = module.so LIB = library LIBINIT = init LIBLIBS = libraries LIBOBJS = objects INSTALL = install-hdr include $(AOLSERVER)/include/Makefile.module AOLserver 4.5: MOD = module DLL = library DLLINIT = init DLLLIBS = libraries OBJS = objects INSTALL = install-dll include $(AOLSERVER)/include/ns.mak -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] urldecoding unicode?
[EMAIL PROTECTED] wrote: I have some data urlencoded by JavaScript's escape function which is turning a Unicode apostrophe into %u2019. AOLserver doesn't recognize this. I can certainly fix this in specific locations, but to handle this universally it seems it this would need to be be fixed in ns_urldecode and/or ns_getform. Any plans to do such a thing? -Titi -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Would you mind submitting a patch for this? Thanks! - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] urldecoding unicode?
[EMAIL PROTECTED] wrote: What I meant was I can fix it on the Tcl side where I'm getting the data, but to make it universal and transparent to Tcl coders such as myself it would have to fixed in the C code of things like ns_urldecode and ns_getform. And I have neither the time nor the chops to take that on right now. So I am asking if anyone had that in mind to do it. If not, it could eventually prove to be a high enough priority for us as we move to more AJAX input, so we would revisit allocating some resources to it at that point. -T. -Original Message- From: Nathan Folkman [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 06, 2006 9:49 AM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] urldecoding unicode? [EMAIL PROTECTED] wrote: I have some data urlencoded by JavaScript's escape function which is turning a Unicode apostrophe into %u2019. AOLserver doesn't recognize this. I can certainly fix this in specific locations, but to handle this universally it seems it this would need to be be fixed in ns_urldecode and/or ns_getform. Any plans to do such a thing? -Titi -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Would you mind submitting a patch for this? Thanks! - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Understand. Would you mind filing it as a bug in SourceForge so we can track it? Thanks! - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] does aolserver run on IBM server
[EMAIL PROTECTED] wrote: Is there anybody on this mailing list If I get in troubles, I’m afraid the other zones on the mainframe crashes as well as mine J I’ll give a shot anyway. That the beauty of an attitude behavior!!! I just need to do backups before Best, iuri *From:* Iuri Sampaio [mailto:[EMAIL PROTECTED] *Sent:* Wednesday, November 29, 2006 4:38 PM *To:* 'AOLSERVER@LISTSERV.AOL.COM' *Subject:* does aolserver run on IBM server Hi, I’d like to know if it’s possible to install aolserver on a IBM S390 generation 5 without pain. I have this mainframe and I’m not quite sure if it’s safe to have aolserver on it The link bellow has the specs. About it http://www-03.ibm.com/servers/eserver/zseries/library/specsheets/pdf/g2214221.pdf iuri -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.430 / Virus Database: 268.14.19/556 - Release Date: 11/28/2006 3:22 PM -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.430 / Virus Database: 268.15.3/562 - Release Date: 12/1/2006 1:12 PM -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Sorry for the delayed response. I'm not aware of anyone testing AOLserver on the platform you describe, and unfortunately do not have access to a similar box to test. I'd be happy to try and help you off list if you run into any issues however. Sorry I can't be of more help! - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] errors compiling nsmysql on osx tiger
[EMAIL PROTECTED] wrote: Has anyone been able to compile the nsmysql driver on os x tiger 10.4.8 / aolserver 4.5 / mysql 5? I've tried on intel and ppc, neither work. The result of make is as follows: gcc -pipe -I/usr/local/mysql/include -Os -Wall -Wno-implicit-int -fno-strict-aliasing -fno-common -I/usr/local/aolserver/include -I/usr/local/aolserver//include -DNO_CONST -DNO_VALUES_H=1 -DHAVE_LIMITS_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_PARAM_H=1 -DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1 -DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_PTHREAD_ATFORK=1 -DTCL_THREADS=1 -DHAVE_COREFOUNDATION=1 -DMAC_OSX_TCL=1 -DTCL_WIDE_INT_TYPE=long\ long -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 -DHAVE_STRSTR=1 -DHAVE_STRTOL=1 -DHAVE_STRTOLL=1 -DHAVE_STRTOULL=1 -DHAVE_TMPNAM=1 -DHAVE_WAITPID=1 -DHAVE_GETPWUID_R_5=1 -DHAVE_GETPWUID_R=1 -DHAVE_GETPWNAM_R_5=1 -DHAVE_GETPWNAM_R=1 -DHAVE_GETGRGID_R_5=1 -DHAVE_GETGRGID_R=1 -DHAVE_GETGRNAM_R_5=1 -DHAVE_GETGRNAM_R=1 -DHAVE_MTSAFE_GETHOSTBYNAME=1 -DHAVE_MTSAFE_GETHOSTBYADDR=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_GMTIME_R=1 -DHAVE_LOCALTIME_R=1 -DHAVE_TM_GMTOFF=1 -DHAVE_ST_BLKSIZE=1 -DSTDC_HEADERS=1 -DHAVE_SIGNED_CHAR=1 -DHAVE_PUTENV_THAT_COPIES=1 -DHAVE_LANGINFO=1 -DHAVE_COPYFILE=1 -DHAVE_LIBKERN_OSATOMIC_H=1 -DHAVE_OSSPINLOCKLOCK=1 -DHAVE_PTHREAD_ATFORK=1 -DUSE_VFORK=1 -DTCL_DEFAULT_ENCODING=\utf-8\ -DTCL_LOAD_FROM_MEMORY=1 -DHAVE_AVAILABILITYMACROS_H=1 -DHAVE_FTS=1 -DHAVE_SYS_IOCTL_H=1 -DHAVE_SYS_FILIO_H=1 -DPACKAGE_NAME=\\ -DPACKAGE_TARNAME=\\ -DPACKAGE_VERSION=\\ -DPACKAGE_STRING=\\ -DPACKAGE_BUGREPORT=\\ -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_TIMEGM=1 -DHAVE_DRAND48=1 -DHAVE_RANDOM=1 -DHAVE__NSGETENVIRON=1 -DHAVE_GETADDRINFO=1 -DHAVE_GETNAMEINFO=1-c -o mysql.o mysql.c mysql.c: In function 'Ns_MySQL_OpenDb': mysql.c:133: warning: unused variable 'len' mysql.c:133: warning: unused variable 'y' mysql.c:133: warning: unused variable 'x' mysql.c:125: warning: unused variable 'rc' mysql.c: At top level: mysql.c:15: warning: 'rcsid' defined but not used /usr/local/aolserver/bin/tclsh8.4 /usr/local/aolserver/bin/nsremove.tcl libnsmysql.dylib gcc -pipe -dynamiclib -install_name /usr/local/aolserver/lib/libnsmysql.dylib -flat_namespace -undefined suppress -o libnsmysql.dylib mysql.o \ -L/usr/local/aolserver/lib -lnsd -lnsthread -L/usr/local/aolserver//lib -ltcl8.4 -lpthread -framework CoreFoundation /usr/local/aolserver/bin/tclsh8.4 /usr/local/aolserver/bin/nsremove.tcl nsmysql.so ld -shared -flat_namespace -undefined suppress -o nsmysql.so -R/usr/local/mysql/lib -L. -lnsmysql -L/usr/local/aolserver/lib -lnsd -lnsthread -L/usr/local/aolserver//lib -ltcl8.4 -lpthread -framework CoreFoundation ld: unknown flag: -shared make: *** [nsmysql.so] Error 1 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. I have, but I have to run, so I'll post the modified Makefile tomorrow. - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] only one connection thread
[EMAIL PROTECTED] wrote: Following up on Stuart's call for clarification, can someone please send some out some documentation, or point us to it, where we can figure out how to use the new ns_limits/ns_pools functionality in 4.5. We just used 4.5 in production and aolserver was limited to 100 ESTABLISHED tcp connections, no matter what we tried. We tried to change the old settings, changing the keepalivetimeout to 0 and messing with: ns_param connsperthread ns_param maxconnections ns_param maxthreads but not matter what we had in our configuration, nothing helped us get any more simultaneous tcp connections. Any help would be greatly appreciated. Or, maybe we should just go back to 4.0.10? Brad On Thu, 10 Aug 2006 20:59:46 +0100, Stuart Children [EMAIL PROTECTED] wrote: Stuart Children wrote: Whilst playing with a new 4.5.0 build, I suddenly noticed that I was only getting one concurrent connection thread running - eek! OK, so this is an affect of the new limits code. It's more specific than I initially described - it's one concurrent connection per specific url, where I believe url actually means url+at least some headers (which is slightly surprising, though I can see people having a requirement for it both ways). Due to the lack of response, I've no idea whether this is the intentional default behaviour - but it would seem to merit a mention in the docs if so. Unfortunately I'm about to leave for an extended holiday, and I've been pretty busy with other matters in my last few days at work, so investigating further how the new limits/pools stuff is actually working (and how one is intended to use it) will have to wait. If anyone can fill in the gaps whilst I'm away that would be great, but I just wanted to let people know I probably won't be replying for three weeks... holiday = no computers. :) Cheers -- Stuart -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. = -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Any overview of the new commands can be found in the release notes: http://aolserver.cvs.sourceforge.net/aolserver/aolserver/RELEASE?view=markup Your best bet is to just look at the source: aolserver/nsd/limits.c aolserver/nsd/pools.c Here's also a few additional notes I had been working on, although probably not that helpful: Ns pools From Office Wiki Thread pools can be configured in either application code or the AOLserver configuration. The following example will set up a thread pool for all GET requests to /slow/* allowing only a maximum of 2 connection threads to be allocated. Additional requests will be queued. ns_pools set slow -maxthreads 2 ns_pools register slow server1 GET /slow/ There are a few stats commands available as well: server1:nscp 1 ns_server threads slow {min 0} {max 2} {current 2} {idle 0} {stopping 0} server1:nscp 2 ns_pools list slow default error server1:nscp 3 ns_pools get slow Ns limits From Office Wiki In addition to creating and registering thread pools, you can also specify the following limits: maxrun maxwait maxupload timeout minthreads 0 maxthreads 2 idle 0 current 2 maxconns 2 queued 8 timeout 5 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Tcl interpreter bloating
[EMAIL PROTECTED] wrote: This morning my server complained: unable to alloc 2169111 bytes How large was the nsd process when it crashed? and thus exited. So, I'm thinking that malloc fragmentation may very well be the problem. I read recently that the connsperthread parameter no longer works in aolserver. I believe this parameter destroys a Tcl interpreter and recreates it, yes?. This technique seems to me a much better way to deal with tcl memory fragmentation than experimenting with a different malloc (ie, it's parallel to the way Apache deals with memory bloat in mod_perl). I'm trying (right now on the live server) to do a manual version of connsperthread, using ns_markfordelete and a per-interpreter counter, cleaning up an interpreter after 1000 uses. I'll be curious to see if that fixes the bloat problem. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Tcl interpreter bloating
[EMAIL PROTECTED] wrote: So did setting connsperthread take care of the bloating? I keep seeing assertions that memory use by AOLserver should level off at some steady state - perhaps controlled by this parameter. But I have yet to see any one come back and say I changed X to Y and now my server doesn't behave like it has a memory leak. FYI, BookMooch.com bloats to 2gigs within 4 days of use and needs to be restarted. I'm hoping that the problem is with the berkeleydb driver AOLserver driver, and am going to dry another database connectivity technique (RMI) to see if that fixes it. If not... -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Your problem is most likely the Tcl memory allocator, and fragmentation that can occur within the pools. There have been some posts detailing these issues a few months ago I think. One thing to try is building Tcl so that it doesn't use the threaded-allocator, and to instead try something like Hoard or Google's tcmalloc. Now all that aside, is it really a big deal to do rolling restarts? - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] More config questions
[EMAIL PROTECTED] wrote: On 9/12/06, Nathan Folkman [EMAIL PROTECTED] wrote: They are still supported. The base.tcl was supposed to be a simpler and more bare bones configuration example. The plan was to add other smaller configuration examples, such as the nscp.tcl which could then be combined with the base.tcl. Never finished - sorry! I just did a check of the 4.5 code and I can't find anywhere where it is reading maxthreads or minthreads from the config.tcl. Changing any of these settings has no effect. ns_section ns/server/server1 ns_param maxthreads 20 ns_param minthreads 5 ns_param maxconns 20 Danie These options have moved to the new ns_pools interface: [1153ahost147:~/cvs/aolserver_head/nsd] shmooved% grep maxconns * ... pools.c:-maxthreads, -minthreads, -maxconns, -timeout, NULL ... - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] internal server redirects
The segfault is obviously a bug... I think what's being referring to is the behavior that allows code to continue executing after an ns_returnredirect unless an ns_adp_abort is explicitly specified. - n [EMAIL PROTECTED] wrote: And the segfaulting when you do this? Also a feature? I'd call it a bug. Nothing you can do in an adp, scheduled proc, etc, etc (plain tcl) should cause the server to crash. Rusty Titi Ala'ilima wrote: It's not a bug, it's a feature. Seriously. Sometimes you want to return a response to the connection and then continue processing afterwards. Or at least, sometimes _I_ want to do that. :-) -Titi -Original Message- From: Daniel P. Stasinski [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 06, 2006 2:10 PM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] internal server redirects it still crashes. Ok, I just found this on the new docs NOTE: The script does not end at the time this command is invoked. ns_adp_abort or ns_adp_return should be called to end script processing after ns_returnredirect This does stop the crash but I would still consider it to be a bug. Daniel -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver's documentation woes and its future
[EMAIL PROTECTED] wrote: (Apologies for the long email ahead ... but, I think it's worth a quick read.) On 2006.09.05, Rick Gutleber [EMAIL PROTECTED] wrote: Support for more popular languages (come on, let's say it together, I know it's hard, but Tcl is not popular) is probably the most useful long-term technical change that can be made. Of course, the more popular languages weren't implemented with embeddability (embedibility?) in mind. Tcl is one of the only mature languages that can be safely embedded in a multi-threaded application. Specifically, what I mean is, Tcl is one of a few languages that can have multiple interpreters in the same process space executing concurrently in multiple threads. Python can be embedded, but according to this: http://docs.python.org/api/api.html 8.1 Thread State and the Global Interpreter Lock http://docs.python.org/api/threads.html The Python interpreter is not fully thread safe. In order to support multi-threaded Python programs, there's a global lock that must be held by the current thread before it can safely access Python objects. [...] A global lock? There goes scalability and the benefits of embedding Python. (You'd be better off running it in a separate process ala FastCGI, AOLserver would gain no performance by embedding it, only risk.) Same problem with Ruby: http://metaeditor.sourceforge.net/embed/ ruby is not thread-safe. If you embed ruby into a multi-threaded application then you will need to semaphore protect all access to ruby. Warning: Insufficient semaphore protection can be hard debuggable. So, they're pointing out the need for a global lock, too. What's worse, the Ruby interpreter uses global variables to manage interpreter state, so even if you did embed it and use a global lock, you couldn't spin up two instances of the interpreter in the same process, anyway! PHP isn't even worth considering, really: Re: [PHP-DEV] Re: PHP and Apache2 http://marc.theaimsgroup.com/?l=php-devm=108736540021355w=2 The major feature that draws people to Apache2 is threading. [...] However, on UNIX there are a lot of basic libraries where thread safety is an unknown. [...] And yes, you could use the prefork mpm with Apache2 to avoid the threading, and yes you could use a standalone fastcgi mechanism to avoid the threading, but you are going out of your way to avoid the defining characteristic of the web server you have decided to use. Rasmus makes it very clear that the reason PHP is so popular and powerful is its large list of third-party extensions that add a lot of out of the box functionality for PHP developers. However, it's that same attribute of PHP that keeps it from playing nicely, embedded in a multi-threaded application. Lets not forget this, too: http://developers.slashdot.org/developers/06/07/28/0552240.shtml Jani Taskinen, one of the lead developers of the Zend Engine (the engine that powers PHP), as well as a lead developer for the thread safety system and other core components of the PHP project, has quit in a relatively cryptic message to the php-internals mailing list. Jani has been involved with PHP for about 6 years and his loss will undoubtedly be a big blow for the PHP project. It'll be interesting to see where the future of PHP takes it. Now, lets look at Perl: http://search.cpan.org/dist/perl/pod/perlembed.pod Now suppose we have more than one interpreter instance running at the same time. This is feasible, but only if you used the Configure option -Dusemultiplicity or the options -Dusethreads -Duseithreads when building perl. [...] Using -Dusethreads -Duseithreads rather than -Dusemultiplicity is more appropriate if you intend to run multiple interpreters concurrently in different threads, because it enables support for linking in the thread libraries of your system with the interpreter. Oh, nice. I don't see any mention of using a globally shared lock to prevent two Perl interpreters running concurrently. (This could just be an omission from the docs, but I have the feeling it's not the case.) So, of the languages more popular than Perl being Python, Ruby, PHP and Perl ... Perl is the only suitable candidate for replacing Tcl in AOLserver. Considering that Perl is no longer the flavor of the day and has been replaced by Python and Ruby amongst the language geeks and PHP amongst the unwashed masses ... who really wants to go replacing AOLserver's embedded Tcl with embedded Perl? How many months or years before people see Perl like they see Tcl today (not popular, etc.)? The irony here is so few people have even heard of Tcl that if it was evangelized and marketed correctly, it could be brought to the marketplace as a new language to replace Ruby and Python and PHP. Write a slew of books, release a mound of free Tcl code and add simple
Re: [AOLSERVER] AOLserver's documentation woes and its future
[EMAIL PROTECTED] wrote: On 5 Sep 2006, at 17:57, Titi Ala'ilima wrote: 1) AOLserver probably needs a new name. Something that uses the NS initials would be ideal so that all those ns_* commands actually make sense again. Could we resurrect the NaviServer name? Already taken! (OpenSource fork-without-anyone-from-AOL-interfering of AOLserver) Maybe we should all focus on NaviServer going forward? Those guys have done a good job backporting work that's been done to the AOLserver head. Just a thought. - n Bas. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
An alternate strategy to exec'ing is to instead expose the low-level C/C++ APIs as Tcl commands. If the code in question isn't thread safe, you can create a library that you would load from the nsproxy, which if you remember, is basically just a single-threaded tclsh that is fork'd and exec'd from the main nsd process. http://www.imagemagick.org/www/magick-wand.html - n [EMAIL PROTECTED] wrote: I use exec for Image Magick, though not in a high-concurrency environment. I wonder how Photo.net deals with this; I always understood they use Image Magick this way. Maybe they did in the beginning and have now switched to a module? Does anyone have experience with using TclMagick inside AOLserver? Cheers, Bas. On Wednesday, August 30, 2006 4:29, Jeff Hobbs said: Tom Jackson wrote: On Saturday 19 August 2006 21:07, Tom Jackson wrote: On Saturday 19 August 2006 19:12, Hossein Sharifi wrote: (although I do plan to fix the incorrect usage of exec as well) So I've never heard that you can't use exec from AOLserver. How is this supposed to be done? Where was this discussed? What would an error look like? Sorry to respond to my own last post, but I've been trying to find out more about this problem of using tcl exec. As far as I can tell, the problem is trying to use fork (in some tcl extension that adds this) and not calling exec in the child process. Is this even possible in exec to not call exec? Does anyone know the status of this problem in the tcl community, from what I have read, the solutions would not be good for AOLserver. The best rundown, with links, is here: http://aspn.activestate.com/ASPN/Mail/Message/activetcl-dev/2848076 Haven't written that thread, what else more specifically do you want to know? ActiveTcl has been shipping thread-enabled for the last several releases. There are some issues with non-core packages (and Tk itself, but more recently resolved), but it has worked well in general. OS X also is thread-built, and even has pthread_atfork usage on Tiger, but it is slightly prone to issues, and OS X has a different fork() implementation than some other common unix variants. You are correct that the main problem is in trying to fork when not closely followed by exec*(). Jeff -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver tuning thru nstelemetry ?
Couple of general things about the Tcl threaded memory allocator (zippy) that might help better explain what you are seeing... - The Tcl memory allocator is a x2 allocator optimized for lock avoidance between threads. This results in very high performance, but comes at the cost of almost x3 overhead for allocations. - There is no garbage collection mechanism provided by the Tcl threaded allocator for pool allocations. Once memory is allocated, it flows back and forth between thread memory pools and the shared memory pool. The exception are very large allocations (the exact threshold escapes me at the moment.) These large allocations are mmap'd directly, and free'd back to the system. - Over extended runs, the memory can tend to get fragmented to the point where the allocator is unable to effectively coalesce smaller blocks together, causing additional memory to be allocated from the system. - In general memory stays tied up in per-thread caches unless thread cleanup occurs, at which point the memory is returned to the shared pool. So if you have a large number of threads which do things like sit in while 1 loops, you could be holding on to large amounts of memory depending on any number of factors. - If you have third-party code which isn't using Tcl_Alloc() and Tcl_Free(), then you aren't using the Tcl threaded allocator. We actually spent a great deal of time trying to tweak the allocator to work around these issues, but in the end the results were inconclusive, and we abandoned this work: http://aolserver.cvs.sourceforge.net/aolserver/zippy2/ 22 /* 23 * Design summary: 24 * 25 * This allocator is based on the legacy single-threaded Tcl allocator 26 * with modifications necessary to support additional features: 27 * 28 * 1. Avoid lock contention: Storage requests smaller than a system 29 * page are allocated with BinAlloc which returns pre-allocated blocks 30 * from per-thread caches. Most small requests can thus be satisfied 31 * without acquiring or releasing mutex locks. To moderate overall 32 * memory growth, bins have a high water mark which defaults to 2x the 33 * number of blocks per system page. When available blocks exceed this 34 * limit, half of the free blocks are moved to a separate shared cache. 35 * When no blocks are available in a per-thread cache, threads first 36 * check the shared cache for these free blocks before calling PageAlloc 37 * to allocate a new page. All free blocks are moved to the shared 38 * cache before a thread exists. In this way, a long running process 39 * should see continual movement of small blocks between the per-thread 40 * and shared caches. 41 * 42 * 2. System allocation for large storage requests: The allocator has a 43 * configurable storage size threshold over which allocations are 44 * performed with system-allocation based MapAlloc and MapFree direclty, 45 * bypassing the malloc-based routines. This has the benefit of 46 * immediately returning to the system very large storage requests, 47 * avoiding the possibility the malloc-based routines holding onto the 48 * large regions. Many malloc-based implementations already support this 49 * operation for some or most allocations so this capability may be 50 * redundant with your particular system. 51 * 52 * 3. Ability to return unused blocks to the system: Cached blocks are 53 * obtained by splitting system pages allocated with PageAlloc. The 54 * alloc reclaim command can be used to sort blocks by their 55 * cooresponding pages and returned complete pages to the free list. 56 * These pages can then be returned to the system with the alloc 57 * freepages command. A more aggressive reclaim is possible with the 58 * alloc wreclaim command which will force other threads which call 59 * allocation routines to dump all available blocks and wait for the 60 * reclaim operation to complete. 61 * 62 * 4. Activity monitoring: For all storage requests, even those beyond 63 * the small blocks in per-thread caches, numerous counters are 64 * maintained on per-thread and a summary basis. The shared cache is 65 * used to accumulate counters for all threads which have previously 66 * exited. In this way, the current and past state of all allocations 67 * for can be summarized assuming the various counters do not overflow 68 * in the course of a long running process. These counters are available 69 * through the alloc binstats, slotslots, and info options or the 70 * the Tcl_GetMemoryInfo procedure. 71 * 72 * 5. Tcl_Obj allocation: Allocations for Tcl_Obj's are handled as a 73 * special case, returning the memory without the normal rounding to the 74 * nearest block size and without a preceding Block structure. 75 * 76 * 6. Tunable parameters: Environment variables can be used to tune the 77 * sizes and features of
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
Just saw this on Digg - pretty hip! http://www.cit.gu.edu.au/~anthony/graphics/imagick6/ - n [EMAIL PROTECTED] wrote: An alternate strategy to exec'ing is to instead expose the low-level C/C++ APIs as Tcl commands. If the code in question isn't thread safe, you can create a library that you would load from the nsproxy, which if you remember, is basically just a single-threaded tclsh that is fork'd and exec'd from the main nsd process. http://www.imagemagick.org/www/magick-wand.html - n [EMAIL PROTECTED] wrote: I use exec for Image Magick, though not in a high-concurrency environment. I wonder how Photo.net deals with this; I always understood they use Image Magick this way. Maybe they did in the beginning and have now switched to a module? Does anyone have experience with using TclMagick inside AOLserver? Cheers, Bas. On Wednesday, August 30, 2006 4:29, Jeff Hobbs said: Tom Jackson wrote: On Saturday 19 August 2006 21:07, Tom Jackson wrote: On Saturday 19 August 2006 19:12, Hossein Sharifi wrote: (although I do plan to fix the incorrect usage of exec as well) So I've never heard that you can't use exec from AOLserver. How is this supposed to be done? Where was this discussed? What would an error look like? Sorry to respond to my own last post, but I've been trying to find out more about this problem of using tcl exec. As far as I can tell, the problem is trying to use fork (in some tcl extension that adds this) and not calling exec in the child process. Is this even possible in exec to not call exec? Does anyone know the status of this problem in the tcl community, from what I have read, the solutions would not be good for AOLserver. The best rundown, with links, is here: http://aspn.activestate.com/ASPN/Mail/Message/activetcl-dev/2848076 Haven't written that thread, what else more specifically do you want to know? ActiveTcl has been shipping thread-enabled for the last several releases. There are some issues with non-core packages (and Tk itself, but more recently resolved), but it has worked well in general. OS X also is thread-built, and even has pthread_atfork usage on Tiger, but it is slightly prone to issues, and OS X has a different fork() implementation than some other common unix variants. You are correct that the main problem is in trying to fork when not closely followed by exec*(). Jeff -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
[EMAIL PROTECTED] wrote: On Tuesday 29 August 2006 20:29, Jeff Hobbs wrote: You are correct that the main problem is in trying to fork when not closely followed by exec*(). But with Tcl exec, you never call fork, screw around and then call exec*(), do you? Is this an issue for C level tcl? Maybe the issue is that fork and exec are separate calls, and something unplanned could happen between the two, and pthread_atfork makes the two calls atomic. (And maybe sometimes even this isn't enough?) Obviously we can now use nsproxy to call exec. I'm not sure if it works for every use of exec inside a full AOLserver, but it would be pretty close. tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. What is it you are trying to exec? - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
[EMAIL PROTECTED] wrote: On Aug 30, 2006, at 8:56 AM, Nathan Folkman wrote: What is it you are trying to exec? From OpenACS we mostly just exec Imagemagick. I can't think of anything else. janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. So the easy way to do this would be to exec from the nsproxy. Better option might be to wrap the C API, although I'm not entirely sure whether or not Imagemagick is truly thread-safe. Even if it is not, you'd probably still see better performance using the wrapped C API from the nsproxy. - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
[EMAIL PROTECTED] wrote: On Wednesday 30 August 2006 10:32, Nathan Folkman wrote: [EMAIL PROTECTED] wrote: On Aug 30, 2006, at 8:56 AM, Nathan Folkman wrote: What is it you are trying to exec? From OpenACS we mostly just exec Imagemagick. I can't think of So the easy way to do this would be to exec from the nsproxy. Better option might be to wrap the C API, although I'm not entirely sure whether or not Imagemagick is truly thread-safe. Even if it is not, you'd probably still see better performance using the wrapped C API from the nsproxy. I've loaded imagemagick into AOLserver: it was much slower than exec'ing the command line tools. tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Wow, that's surprising! Then exec'ing from the nsproxy is probably your best bet. - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
[EMAIL PROTECTED] wrote: On Aug 30, 2006, at 10:32 AM, Nathan Folkman wrote: So the easy way to do this would be to exec from the nsproxy. Better option might be to wrap the C API, although I'm not entirely sure whether or not Imagemagick is truly thread-safe. Even if it is not, you'd probably still see better performance using the wrapped C API from the nsproxy. Is this concern something new for 4.5? Because OpenACS has been exec'ing Imagemagik, sqlplus and plsql for years now with no apparent problems. Have we just been really lucky? janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. If it ain't broke don't fix it. ;-) But if you're running into memory issues, the nsproxy approach I mentioned can help. Another nice thing in general about running code from within the nsproxy is that you can easily kill of the nsproxy processes which can help you reclaim memory that otherwise might be tied up in the Tcl threaded allocator shared pools. This is more of a general side affect, and not really relevant to the exec'ing discussion... - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is tcl exec _really_ bad with threads
[EMAIL PROTECTED] wrote: On Saturday 19 August 2006 21:07, Tom Jackson wrote: On Saturday 19 August 2006 19:12, Hossein Sharifi wrote: (although I do plan to fix the incorrect usage of exec as well) So I've never heard that you can't use exec from AOLserver. How is this supposed to be done? Where was this discussed? What would an error look like? Sorry to respond to my own last post, but I've been trying to find out more about this problem of using tcl exec. As far as I can tell, the problem is trying to use fork (in some tcl extension that adds this) and not calling exec in the child process. Is this even possible in exec to not call exec? Does anyone know the status of this problem in the tcl community, from what I have read, the solutions would not be good for AOLserver. The best rundown, with links, is here: http://aspn.activestate.com/ASPN/Mail/Message/activetcl-dev/2848076 tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Something else to watch out for is the side effect of doing a fork() then exec() from the main nsd process. There's a small window of time where you can have double the memory of your main nsd process tied up while the main nsd process space is copied. So, for example, if your nsd runs really hot at say 1G, then you'd need at least an extra 1G, and maybe more if you have multiple threads all trying to fork() and exec() something. We've seen this with things like the nsdb code and the new nsproxy code. Generally this isn't a problem, but something to be aware of. That said, a much better strategy is to immediately fork off an nsproxy and do all further execs from within the nsproxy (tclsh) which is considerably smaller then the main nsd process. Just curious, what is it that you are looking to exec? - n - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] XOTlc and AOLServer 4.5
[EMAIL PROTECTED] wrote: Hi! I'm trying to use the throttle xotcl package with AOLServer 4.5, which works in 4.0.10. When I call a page, I get the error invalid command name ::xotcl::Object from the ns_register_filter postauth GET * throttle page filter. The xotcl namespace is imported in a tcl library file at startup, I even tried to import it with ns_eval. Has something changed in the way AOLServer imports namespaces at startup? Thanks, wiwo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Try adding the following to your Tcl library which gets sourced at startup: ns_ictl package require xotcl See also: http://aolserver.cvs.sourceforge.net/aolserver/aolserver/doc/ns_ictl.n?view=markup - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] question about libnsd loading in tclsh
Locks are not shared with the nsd process. The libnsd library came about as a result of folks wanting some of the AOLserver core Tcl APIs from within stand alone tclsh processes. You might also want to look at the new nsproxy module. That code allows you to start a pool of tclsh processes that you can send commands to. It's a great way to deal with code which is not thread safe, since each tclsh, by default, is single threaded. The tclsh processes are fork'd and exec'd from the main nsd process. Example: ns_proxy pool - nsd (MT) tclsh (ST) tclsh (ST) tclsh (ST) - n -Original Message- From: [EMAIL PROTECTED] To: AOLSERVER@LISTSERV.AOL.COM Sent: Tue, 15 Aug 2006 5:52 PM Subject: [AOLSERVER] question about libnsd loading in tclsh In the 4.5 release docs, it was announced: libnsd: The AOLserver library now includes an entry point suitable for loading into an ordinary, thread-enabled, tclsh, e.g.,: # tclsh % load /usr/local/aolserver/lib/libnsd.so % ns_time Does this loaded libnsd.so share locks, such as ndb_handles and semaphores with an already running nsd? I assume not, but thought I'd ask... What I'm trying to do is run command-line tcl commands that would access my berkeleydb database while aolserver is still running, and if two separate processes are writing the same db file, it gets corrupted. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. Check out AOL.com today. Breaking news, video search, pictures, email and IM. All on demand. Always Free. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] cron (ie, timed) jobs in aolserver
Have you looked at the "ns_schedule_proc" APIs? Example: http://panoptic.com/wiki/aolserver/Ns_schedule_daily - n -Original Message- From: [EMAIL PROTECTED] To: AOLSERVER@LISTSERV.AOL.COM Sent: Tue, 15 Aug 2006 6:48 PM Subject: [AOLSERVER] cron (ie, timed) jobs in aolserver Anyone have any advice on how to run tasks on a scheduled basis within the aolserver process? Currently, I use cron to run "lynx http://bookmooch.com/nightlyjobs.adp", and that works fine, but was wondering if there was an all-aolserver way, that was more elegant. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. Check out AOL.com today. Breaking news, video search, pictures, email and IM. All on demand. Always Free. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] cvs head build problem
The plan was to roll all that back so we could get to a simple, and some would argue, more standard build style: - autoconf - configure - make Not quite there yet though it seems... In the mean time, the README file is accurate, and contains instructions that illustrate how things differ from how the 4.0 and earlier versions are built. - n [EMAIL PROTECTED] wrote: Actually was configure changed back to being the main build? Last time I played with it spent half a night writing a patch until Nate told me to look at the Readme :-) It was changed to doing % /path/to/tclsh nsconfig.tcl Which then calls configure- or has it been updated? P Dossy Shiobara wrote: On 2006.08.07, John Buckman [EMAIL PROTECTED] wrote: The modifications to util/*.tcl stated above seem critical and the CVS head should be so patched. The build is still kinda shaky. In theory, you could build like this (on Debian): $ ./configure TCLSH=/usr/bin/tclsh --with-tcl=/usr/lib/tcl8.4 $ make $ make install And it'll use the path specified in TCLSH as the executable to use to execute the .tcl scripts. I'm not sure I like this: for one, we should probably use autoconf to auto-detect the tclsh binary using the TCL_EXEC_PREFIX from tclConfig.sh or some other magic as a default. That would probably eliminate the most FAQ build issue with 4.5.0 that you ran up against. The pre-3.3 gcc error you encountered I created a patch for and folks have verified it fixes the problem, so I'll commit that now. Thanks again for the feedback and congratulations on BookMooch.com, by the way. I love the Server: header. :-) -- Dossy -- Patrick O'Leary AOL Search Phone: + 1 703 265 8763 Honesty is the best policy, but insanity is a better defense ! -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Fedora 5 and aolserver 4.5.0
Try the following steps (if you haven't already) from the aolserver directory, and see if this works for you: 1. ./configure --with-tcl=/usr/local/tcl/lib --prefix=/usr/local/aolserver --enable-symbols 2. /usr/local/tcl/bin/tclsh8.4 nsconfig.tcl -install /usr/local/aolserver -debug 3. make 4. make install - n [EMAIL PROTECTED] wrote: I could never make 4.0.10 work with FC5 even with a local version of tcl. multiple definition of `_init' shows up no matter what I do. I am now trying to compile 4.5 and the following error is thrown. nsdb nssock nslog nsperm nscgi nscp make: execvp: ./util/nsmakeall.tcl: Permission denied make: *** [build] Error 127 Any ideas? It appears that Fedora and AOL don't like each other. All my other boxes are gentoo and I have no problems. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER]
If you haven't already, would you mind filing this as a bug against 4.5 over at SourceForge? I set up a new category for 4.5 bugs. I'll try to take a look one of these evenings. Also, please provide a fail case. Thanks! - n [EMAIL PROTECTED] wrote: Now... if I visit a bogus url, I *still* receive the default aolserver not found page. If I visit /404.adp directly, I successfully get the sorry not found message. So, the redirects section seems to have no affect at server startup. I've tried a few other methods with the same results. I've gone back and verified that all of this works correctly under 4.0.10. I guess either this is a bug, or just a change in how to handle this. Has anyone else run into this problem? I'd appreciate if anybody could give a bit of time to sorting this out. Yes, I reported this problem a few days ago, and should have mentioned that I was running the CVS head of 4.5, so it looks to me like it may be a genuine 4.5 bug. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] [ANN] AOLserver 4.5.0 released!
The latest Tcl version (8.4.13) works great! Haven't spent any amount of time testing with upcoming Tcl 8.5, but it also should work fine I believe. Not sure exactly what memory issues you are referring to, but if you are building Tcl multi-threaded, then you are using the threaded allocator (former Zippy allocator) which is simple x2 allocator. The allocator is optimized for front-end performance, and delivers pretty much 0% lock contention. This comes at the cost of wasted bytes through bucket size allocations, and also there is currently no garbage collection facilities provided. We recently released a new version of the Tcl memory allocator code which does provide garbage collection, but it was unclear in our testing if that approach would ever really be valid. If high performance is your goal, then the Tcl threaded allocator is probably best for you. Alternately you could try something like Hoard, or even Google's tcmalloc. Hope that helps! - n [EMAIL PROTECTED] wrote: What version of Tcl is recommended for use with AOLserver 4.5.0? Have memory usage/depletion issues been resolved in AOLserver 4.5.0, when using the recommended version of Tcl? Harvey Lunenfeld Internet Corporation [EMAIL PROTECTED] - Original Message - From: Dossy Shiobara [EMAIL PROTECTED] To: AOLSERVER@LISTSERV.AOL.COM Sent: Tuesday, June 27, 2006 3:55 PM Subject: [AOLSERVER] [ANN] AOLserver 4.5.0 released! AOLserver 4.5.0 Released June 27, 2006 On behalf of the AOLserver Team, I have the honor of announcing the latest major release of AOLserver: 4.5.0. AOLserver 4.5.0 is a major upgrade including several new Tcl commands, C API's, and resource management improvements. A summary of changes is provided at the end of this document. WHAT IS AOLSERVER? AOLserver is America Online's Open-Source web server. AOLserver is the backbone of the largest and busiest production environments in the world. AOLserver is a multithreaded, Tcl-enabled web server used for large scale, dynamic web sites. Visit the project's website: http://aolserver.com/ HOW CAN I GET AOLSERVER? Download the source code from SourceForge: http://aolserver.com/downloads/aolserver-4.5.0-src.tar.gz I FOUND A BUG! WHAT DO I DO? File it in the Bug Tracker at SourceForge: http://aolserver.com/sf/bugs IS THERE A MAILING LIST? WHERE'S THE USER COMMUNITY? Yes! There's an announcements-only and a general discussion mailing list. Instructions on how to subscribe are here: http://aolserver.com/lists.php There is also a wiki-web set up for AOLserver: http://aolserver.com/wiki/ CHANGES SINCE LAST RELEASE? AOLserver 4.5 - June 2006 - AOLserver 4.5 is a major upgrade including several new Tcl commands, C API's, and resource management improvements. Contents: New and Updated Binaries New Modules New and Updated Tcl Commands New and Updated C Routines Additional Updates Changes to the Build Process Bug Fixes Known Issues New and Updated Binaries: libnsd: The AOLserver library now includes an entry point suitable for loading into an ordinary, thread-enabled, tclsh, e.g.,: # tclsh % load /usr/local/aolserver/lib/libnsd.so % ns_time This provides the full set of Tcl commands within the AOLserver core although several commands will generate errors for lack of an HTTP request context (e.g., ns_conn). nstclsh: A simple Tcl-shell linked against libnsd. It's equivalent to loading libnsd into an ordinary tclsh as described above. New Modules: --- nszlib: The popular nszlib module written by Vlad Seryakov has been integrated into the core release with modifications to enable the new Ns_Gzip routine. The module also provides the ns_zlib command for compressing and uncompressing strings and files. nsproxy: The nsproxy modules provides the ns_proxy command which enables sending Tcl scripts to a proxy process connected via a pipe. The proxy includes core Tcl commands as well as AOLserver commands from the libnsd library. Evaluating scripts in a proxy process can be used to isolate and/or timeout thread-unsafe or otherwise unsafe 3rd party code. New and Updated Tcl Commands: The following Tcl commands are new or updated from the last release. As few of the man pages have yet to be written, see the source code for usage details and options. ns_pools: The ns_pools command enables configuration of one or more pools of connection processing threads. The pools allow certain requests to be handled by specific threads. This could, for example, ensure multiple long running requests don't block other short running requests. Pools are selected based on method/url pairs similar to the mappings managed by the ns_register_proc
Re: [AOLSERVER] [ANN] AOLserver 4.5.0 released!
I'd like to say categorically yes, but it depends on specifically which issues you were referring to. There have been some misconceptions in the past about AOLserver's memory usage patterns in regards to how the threaded allocator works, and I'm aware of a few leaks in some older versions of Tcl, but to the best of my knowledge these have been addressed in the current Tcl 8.4.13 release. Was there a specific issue you were concerned about? - n [EMAIL PROTECTED] wrote: There were a number of discussions re AOLserver 4.10 memory usage problems, which were associated with at least one version of Tcl. Is it fair to assume that those memory usage problems have been eliminated in AOLserver 4.5.0 when used with Tcl 8.4.1.3? Thanks. Harvey Lunenfeld Internet Corporation [EMAIL PROTECTED] - Original Message - From: Nathan Folkman [EMAIL PROTECTED] To: AOLSERVER@LISTSERV.AOL.COM Sent: Thursday, July 06, 2006 1:50 PM Subject: Re: [AOLSERVER] [ANN] AOLserver 4.5.0 released! The latest Tcl version (8.4.13) works great! Haven't spent any amount of time testing with upcoming Tcl 8.5, but it also should work fine I believe. Not sure exactly what memory issues you are referring to, but if you are building Tcl multi-threaded, then you are using the threaded allocator (former Zippy allocator) which is simple x2 allocator. The allocator is optimized for front-end performance, and delivers pretty much 0% lock contention. This comes at the cost of wasted bytes through bucket size allocations, and also there is currently no garbage collection facilities provided. We recently released a new version of the Tcl memory allocator code which does provide garbage collection, but it was unclear in our testing if that approach would ever really be valid. If high performance is your goal, then the Tcl threaded allocator is probably best for you. Alternately you could try something like Hoard, or even Google's tcmalloc. Hope that helps! - n [EMAIL PROTECTED] wrote: What version of Tcl is recommended for use with AOLserver 4.5.0? Have memory usage/depletion issues been resolved in AOLserver 4.5.0, when using the recommended version of Tcl? Harvey Lunenfeld Internet Corporation [EMAIL PROTECTED] - Original Message - From: Dossy Shiobara [EMAIL PROTECTED] To: AOLSERVER@LISTSERV.AOL.COM Sent: Tuesday, June 27, 2006 3:55 PM Subject: [AOLSERVER] [ANN] AOLserver 4.5.0 released! AOLserver 4.5.0 Released June 27, 2006 On behalf of the AOLserver Team, I have the honor of announcing the latest major release of AOLserver: 4.5.0. AOLserver 4.5.0 is a major upgrade including several new Tcl commands, C API's, and resource management improvements. A summary of changes is provided at the end of this document. WHAT IS AOLSERVER? AOLserver is America Online's Open-Source web server. AOLserver is the backbone of the largest and busiest production environments in the world. AOLserver is a multithreaded, Tcl-enabled web server used for large scale, dynamic web sites. Visit the project's website: http://aolserver.com/ HOW CAN I GET AOLSERVER? Download the source code from SourceForge: http://aolserver.com/downloads/aolserver-4.5.0-src.tar.gz I FOUND A BUG! WHAT DO I DO? File it in the Bug Tracker at SourceForge: http://aolserver.com/sf/bugs IS THERE A MAILING LIST? WHERE'S THE USER COMMUNITY? Yes! There's an announcements-only and a general discussion mailing list. Instructions on how to subscribe are here: http://aolserver.com/lists.php There is also a wiki-web set up for AOLserver: http://aolserver.com/wiki/ CHANGES SINCE LAST RELEASE? AOLserver 4.5 - June 2006 - AOLserver 4.5 is a major upgrade including several new Tcl commands, C API's, and resource management improvements. Contents: New and Updated Binaries New Modules New and Updated Tcl Commands New and Updated C Routines Additional Updates Changes to the Build Process Bug Fixes Known Issues New and Updated Binaries: libnsd: The AOLserver library now includes an entry point suitable for loading into an ordinary, thread-enabled, tclsh, e.g.,: # tclsh % load /usr/local/aolserver/lib/libnsd.so % ns_time This provides the full set of Tcl commands within the AOLserver core although several commands will generate errors for lack of an HTTP request context (e.g., ns_conn). nstclsh: A simple Tcl-shell linked against libnsd. It's equivalent to loading libnsd into an ordinary tclsh as described above. New Modules: --- nszlib: The popular nszlib module written by Vlad Seryakov has been integrated into the core release with modifications to enable the new Ns_Gzip routine. The module also provides the ns_zlib command for compressing and uncompressing strings and files
Re: [AOLSERVER] Building 4.5 on OS X - .dylib vs. .a
How are you building Tcl and AOLserver? - n [EMAIL PROTECTED] wrote: Oh boy, another OS X build problem from me... :( - ranlib: can't open file: /Users/bas/dev/aolserver-4.5.0/lib/libnsthread.a (No such file or directory) make[1]: *** [install-dll] Error 1 make: *** [install-bins] Error 1 geefive:~/dev/src/aolserver-4.5.0 bas$ ls /Users/bas/dev/aolserver-4.5.0/lib libnsthread.dylib libtcl8.4.dylib libtclstub8.4.a tcl8.4 tclConfig.sh geefive:~/dev/src/aolserver-4.5.0 bas$ - As you can see, this is a .dylib on OS X. Anoyone seen this before and/or have a fix? Cheers, Bas. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Scaling at the high end
In terms of distributed key/value pair data structures, there are two additional methods we've used here at AOL: * AV (auto variables) * NV (network variables) AV's are a read-only data structure that under the covers is built on a sorted list for quick access, and small size. The files get built and then distributed to the front-end servers where the updated versions are automatically swapped in by code that looks at the last modification time for the AV file, and also some additional code which maintains reference counts for AV usage across interps. We tend to use RDIST to distribute these files from the server that creates them to the front-ends. NV's are best suited for relatively small data sets which have frequent updates. The classic example is a stock ticker application. Updates are streamed to connected front-ends via persistent DCI_RPC connections between the NV writer and those front-end servers. Both of the mechanisms are similar to how you would use NSV (NaviServer variables) locally, but can be used with a farm of connected front-end hosts. NV's also provide some additional functionality that you don't get with NSV's out the box, for example, a configuration option that will periodically dump the contents from memory to disk, as well as reading the version stored on disk at startup for persistent data storage across restarts. A third approach we use is to use our SOB (small object blaster) servers and APIs to store data. The data can be anything, but most often is a Tcl list. The SOB APIs provide various caching levels, which depending on your application, might help with memory size since you'd only be caching data that was requested, as opposed to loading an entire data set into memory as you would with NSVs, AVs, and NVs. Anyhow, these APIs are part of the nsdci AOLserver module which we will be releasing soon. - n [EMAIL PROTECTED] wrote: On Monday 03 July 2006 08:33, John Buckman wrote: One of the reasons I want to use Berkeley DB is that I'd like every web page string to be a BDB database lookup, allowing wiki-style correcting of strings on a web page (ie, anyone can correct, on the spot, any translated text on any page). SQL just wouldn't work with that design, with a dozen or so db lookups per page. So the design requires a number of queries, too many for responsive performance from an SQL database. However, if you use the BDB database, you will still have to go through some sort of database API, either the one which exists, or a custom designed API. The purpose of the built in API is to share connections across threads and even virtual servers, but it isn't really suited for simple key/value lookup. If the number of key/value pairs is small, everything could be loaded into memory, into an nsv_array. If the number of key/value pairs is large, then the solution is to load pairs into memory only on first use. Maybe a high-use cache could be developed and pre-loaded on server startup. If the cache will eventually grow too large, then ns_cache could be used to expire stale data. Maybe BDB doesn't have the option to unload memory mapped regions. If this is true, then ns_cache would be the better choice to control memory usage. But none of these solutions deals with how the key/value pairs are added to the database and distributed to one or more AOLservers. At some point there has to be a single master list of these pairs. Loading all these into memory, or into a BDB or DJB cdb requires only one SQL query. You can even do one query to write the database on the master, then distribute this with rsync or whatever to replace the old database. Overall, it seems like BDB or cdb would have the same memory requirements of an nsv array, but they require some kind of API to get to this memory. Since it is just key/value pairs, this API overhead is a total waste: you have to use it even when the data is already in memory. tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)
Speaking of SOB, we're working on getting the code to our dci (Digital City) module ready for release to the Open source community. This module contains the following items which folks may find interesting: * SOB - Dossy described this below. Basically it's what we use at AOL to forward cache content. * Network Poll - Provides a simple mechanism to create member facing polls. * Network Tally - Allows one to aggregate counts for keys. We've used this in the past to track things like user views of a particular venue. * Network Comment Boards - Allows one to easily create and add comment boards to your application. * Network NV (network variables) - Similar to nsv, but allows updates to be distributed to many clients. Allows you to update variables on a single NV writer, and have those changes broadcast out to many front-end readers. * Network Eval - Allows you to send Tcl scripts to be evaluated to many other servers. Those are the really big things. They've been at use at AOL since the early Digital City days, and although relatively simplistic in their design - most assume single-writer for instance, they have shown themselves to be quite robust. Stay tuned for the release of the nsdci module! ;-) - n [EMAIL PROTECTED] wrote: On 2006.07.03, John Buckman [EMAIL PROTECTED] wrote: In my experience, many applications that use SQL actually only need key-lookup capability [...] Which is why, at AOL, a large part of the web publishing architecture utilizes Small Object Broker (SOB), which can be set up in a distributed (master-slave style) network based service with client-side read caching with flush-on-write. This lets you simply scale your key-based query infrastructure horizontally. Initially, I think some of the conventional wisdom at the high-scalability end runs counter to what the majority of web developers have discovered and learned because almost any reasonable solution will work in the small which is where most web projects live. In those problem sets, tools and solutions which have a shallower learning curve and offer more functionality out of the box yield faster development times and empower developers to be productive with a smaller skill set. However, there's a line (I couldn't yet tell you exactly where it is), that once you cross it, the characteristics of the problem set changes and the optimal (and sometimes only) solutions become very specialized. It's at this end of the spectrum where AOLserver truly shines, but it's a very small set. John, it's fantastic to see you're actively looking at and working with AOLserver. From the list archives, it looks like your first message came through around 15 Apr 2006. I'm hoping that your interest in AOLserver might mean that the upcoming version of Lyris may replace tclhttpd with AOLserver? Or are you working with AOLserver for something different (i.e., Magnatune)? -- Dossy -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip compression
Don't think so as I believe it relies on some newly added C APIs (Ns_SetGzipProc, Ns_Gzip, etc.), which are only available in AOLserver 4.5. Have you looked at the nszlib AOLserver module? It's available from the AOLserver CVS tree: aolserver/nszlib Hope that helps! - n [EMAIL PROTECTED] wrote: On Monday 26 June 2006 21:47, Nathan Folkman wrote: Yes. In fact, there's a new nszlib AOLserver module as part of the AOLserver 4.5 release. Thanks !!! Can it also be used on/in Aolserver 4.0.10 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip compression
Yes. In fact, there's a new nszlib AOLserver module as part of the AOLserver 4.5 release. Here's the relevant release notes: nszlib: The popular nszlib module written by Vlad Seryakov has been integrated into the core release with modifications to enable the new Ns_Gzip routine. The module also provides the ns_zlib command for compressing and uncompressing strings and files. C APIs: Ns_Gzip: Ns_SetGzipProc: Routine to gzip-compress a buffer if a compression function has been registered with a call to Ns_SetGzipProc. The new nszlib module registers such a callback. Tcl APIs: ns_zlib ns_zlib option ?arg? ns_adp_ctl gzip ?bool?; # Gzip output if enabled in server? Also be sure to check out the README file (aolserver/nszlib/README) - n [EMAIL PROTECTED] wrote: Hello, I would like to ask if AOLServer supports gzip compression, similar to Apache's mod_gzip. Thanks, Hamilton -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] New release?
Not sure, I was able to compile the head version on the following box just fine: ~ uname -a Linux ... 2.6.15-1.2054_FC5 #1 Tue Mar 14 15:48:33 EST 2006 i686 athlon i386 GNU/Linux If you are still having issues, would you mind opening a bug ticket with as much information as you can provide? Thanks! - n [EMAIL PROTECTED] wrote: It doesn't compile on 32 bit FC5 as several posts have showed. It is a problem with tcl and threads. I use gentoo, but have 1 FC5 box so I am no Fedora expert. Nathan Folkman wrote: I know it compiles on 32-bit FC5. Haven't tried on 64-bit FC5. - n [EMAIL PROTECTED] wrote: How about the fix for FC5 that stops AOL from compiling. Nathan Folkman wrote: We're finishing up work on the 4.5 release which should address many of these 64-bit issues. Could you please check out the head and try it out? Thanks! - n [EMAIL PROTECTED] wrote: Hello all, Would it be possible to do a new AOLserver release sometime soon? The 4.0.10 version doesn't work on an x86_64 system, and as world moves to 64 bit, this starts to become a bit annoying. I know it is not that hard to fix (copy the offending file from cvs), but this is really going to alienate people from AOLserver. Also, can the problem please get documented? Daniël Mantione -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Maintained?
[EMAIL PROTECTED] wrote: On 2006.06.20, Eduardo Perez [EMAIL PROTECTED] wrote: I would like to know if aolserver continues to be maintained or if you are considering it deprecated and suggest people to move to other server. I am asking because I see no new releases and the https://sourceforge.net/projects/aolserver/ bugtracker seems to have no activity at all even on bugs having patches on them. The short answer is that yes, AOLserver is still maintained and supported, although there's very few people working on it. Jim Davidson and Nate Folkman have been working very hard on wrapping up the changes for AOLserver 4.5.0 and I suspect it'll be tagged/released It's true, just finishing up the release notes... Should have something officially tagged and released in the next couple of days. - n any day now. -- Dossy -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] New nsloopctl AOLserver Module Released
On the heels of the Zippy2 code release, I've also committed a new module called nsloopctl. This module redefines Tcl's for, foreach, and while commands to allow both the gathering of statistics about each loop, and also to provide a mechanism to pause, cancel, and resume a particular loop. http://aolserver.cvs.sourceforge.net/aolserver/nsloopctl/ This module has shown itself to be quite useful as both a debugging tool, and the performance hit is slight enough that you should be able to run in a production environment as well. Hopefully someone will find this useful! Enjoy! ;-) - n -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] New nsloopctl AOLserver Module Released
In AOLserver 4.5 the following new Tcl APIs are provided: ns_for ns_foreach ns_while ns_loop_ctl The usage is exactly the same as the nsloopctl module I checked in, and which I documented in the README file. The only difference between the two is that nsloopctl will actually redefine Tcl's for, foreach, and while commands, while the new AOLserver 4.5 APIs require you to use the new commands explicitly. Forgot we had added it to the core, so I'll remove - sorry about the confusion. ;-) Here's a quick example: server1:nscp 4 ns_thread begindetached while 1 {ns_sleep 1:0} server1:nscp 5 ns_loop_ctl list server1:nscp 6 ns_thread begindetached ns_while 1 {ns_sleep 1:0} server1:nscp 7 ns_loop_ctl list 0 server1:nscp 8 ns_loop_ctl info 0 0 41973248 1149519792:592471 9 running {ns_while 1 {ns_sleep 1:0}} server1:nscp 9 ns_loop_ctl pause 0 1 server1:nscp 10 ns_loop_ctl resume 0 1 server1:nscp 11 ns_loop_ctl cancel 0 1 server1:nscp 12 ns_loop_ctl list If you want to gather statistics about existing for, foreach, and while loops, simply rename those commands and have them instead point to the new commands: for - ns_for foreach - ns_foreach while - while Hope that helps! - n [EMAIL PROTECTED] wrote: Nathan, This is interesting, but I thought I saw this code as part of 4.5, wasn't this committed earlier? Are you going to post examples? It is hard to tell exactly what this does differently than the tcl commands. At some point a tcl error isn't appended to the result, but I don't know if this is your code of copied from tcl. How is it possible to pause/resume a loop? tom jackson On Monday 05 June 2006 06:20, Nathan Folkman wrote: On the heels of the Zippy2 code release, I've also committed a new module called nsloopctl. This module redefines Tcl's for, foreach, and while commands to allow both the gathering of statistics about each loop, and also to provide a mechanism to pause, cancel, and resume a particular loop. http://aolserver.cvs.sourceforge.net/aolserver/nsloopctl/ This module has shown itself to be quite useful as both a debugging tool, and the performance hit is slight enough that you should be able to run in a production environment as well. Hopefully someone will find this useful! Enjoy! ;-) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] New nsloopctl AOLserver Module Released
[EMAIL PROTECTED] wrote: Nathan, So this is server wide control and monitoring. I'm wondering if it would be possible to do something like an incr type control, where you could have a loop and signal it to run once, or some number of times, and then pause. (Oh, wait, maybe the eval subcommand runs the script once?) So I guess I should ask if there is a way to begin a loop in a paused state? ns_loop_ctl eval script is exactly how you would implement something like this. You could imagine pausing the loop and injecting some code that would wait on a condition variable that was set elsewhere. On the examples: it seems like you would have to be pretty lucky to catch a loop executing, this must be for finding loops which are misbehaving and getting out of them without restarting the server. Exactly right - on certain applications we have a number of detached threads which degenerate to while 1 situations, where this technique is useful. We were also curious as to how long each spin of certain loops was taking. It would also be nice if the info subcommand provided the number of spins or loop iterations already performed. This might allow some form of automatic detection and canceling of loops gone wild. It does: loop_id thread_id elapsed_time number_spins state script 0 41973248 1149519792:592471 9 running {ns_while 1 {ns_sleep 1:0}} You could imagine writing your own code to detect these run-away cases. Maybe something like this: proc detectBadLoops {} { set maxSpins 100 foreach loopId [ns_loop_ctl list] { # # Catch to avoid race condition around loops which # may have completed under your feet. # catch { set loopInfo [ns_loop_ctl info $loopId] set spins [lindex $loopInfo 3] if {$spins $maxSpins} { ns_loop_ctl cancel $loopId } } } ns_sleep 5:0 } ns_thread begindetached detectBadLoops ...something like that at least... ;-) Anyway, pretty interesting code. Hopefully to someone other then us! ;-) - n tom jackson On Monday 05 June 2006 08:11, Nathan Folkman wrote: In AOLserver 4.5 the following new Tcl APIs are provided: ns_for ns_foreach ns_while ns_loop_ctl The usage is exactly the same as the nsloopctl module I checked in, and which I documented in the README file. The only difference between the two is that nsloopctl will actually redefine Tcl's for, foreach, and while commands, while the new AOLserver 4.5 APIs require you to use the new commands explicitly. Forgot we had added it to the core, so I'll remove - sorry about the confusion. ;-) Here's a quick example: server1:nscp 4 ns_thread begindetached while 1 {ns_sleep 1:0} server1:nscp 5 ns_loop_ctl list server1:nscp 6 ns_thread begindetached ns_while 1 {ns_sleep 1:0} server1:nscp 7 ns_loop_ctl list 0 server1:nscp 8 ns_loop_ctl info 0 0 41973248 1149519792:592471 9 running {ns_while 1 {ns_sleep 1:0}} server1:nscp 9 ns_loop_ctl pause 0 1 server1:nscp 10 ns_loop_ctl resume 0 1 server1:nscp 11 ns_loop_ctl cancel 0 1 server1:nscp 12 ns_loop_ctl list If you want to gather statistics about existing for, foreach, and while loops, simply rename those commands and have them instead point to the new commands: for - ns_for foreach - ns_foreach while - while -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLServer 4.0.10 and Mac OS X 10.4.6
SourceForge recently updated their CVS servers: http://sourceforge.net/cvs/?group_id=3152 You'll need to check out fresh. - n [EMAIL PROTECTED] wrote: Hi again, I have try all day to checkout from the sourceforge CVS but the connection is refused. Is there something wrong with sourceforge ? Herick Hérick Moniz, directeur du développement technologique [EMAIL PROTECTED] __ Disques RSB Inc. 8400, Côte-de-Liesse Montréal, QC H4T 1G7 Téléphone : 514.342.8511 poste 228 Télécopieur : 514.342.0401 Sans-frais : 1.800.361.8153 On Jun 2, 2006, at 5:17 PM, Nathan Folkman wrote: Check out, or update, your code from the HEAD. - n [EMAIL PROTECTED] wrote: Hi, Thank again for your help. How could I get this fix ? Is it working ? Herick On Jun 2, 2006, at 3:58 PM, Jim Davidson wrote: I just checked in the fix which is really just to disable the configure check for poll on OS/X, thus not setting the HAVE_POLL cdef and defaulting to the poll emulation code used on OS/X in the past. So, update, re-config, and rebuild -- should work now. For those curious, it appears that poll on OS/X will return POLLOUT, i.e., ready to write without blocking, but when spinning quickly to send a large file it appears to lie and the sendmsg following the poll ends up with an EAGAIN anyway. -Jim On Jun 1, 2006, at 5:20 PM, Nathan Folkman wrote: Was talking with Jim about this issue earlier today. Seems to be an issue with OS X supplied poll(). We should have a fix checked in either today or tomorrow - stay tuned! - n [EMAIL PROTECTED] wrote: Any thing I could look on my side ? I got this only on Mac OS X 10.4.X. Everything is fine on Mac OS X 10.3.9 Could it be related to GCC4 the default compiler on 10.4.X ? Thanks for your help ! Herick On Jun 1, 2006, at 10:20 AM, Nathan Folkman wrote: Interesting... Good news is that I can definitely reproduce. :-) I'll take a look. Interestingly the response headers seem to be correct: -rw-r--r-- 1 shmooved shmooved 173551012 Apr 21 10:41 Desktop/Inbox/cvs.zip http://10.0.1.3:8000/cvs.zip GET /cvs.zip HTTP/1.1 Host: 10.0.1.3:8000 User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive HTTP/1.x 200 OK Last-Modified: Fri, 21 Apr 2006 14:41:41 GMT MIME-Version: 1.0 Date: Thu, 01 Jun 2006 13:44:16 GMT Server: AOLserver/4.5.0a Content-Type: application/zip Content-Length: 173551012 Connection: keep-alive I'll post more once I find something. Thanks! - n [EMAIL PROTECTED] wrote: Hi, I got a strange bug with aolserver on the latest version of Mac OS X, on PPC or Intel. Everything compile fine, with gcc4 or gcc 3.3, and the software seems to run fine but I am not able to transfer file larger then 100 Kb. The transfer start but stop without any error or warning. I have try to patch conn.c as suggest for gcc4 without success. I have scan google for a fix without any help ! Any suggestions ? Thanks Herick Hérick Moniz, directeur du développement technologique [EMAIL PROTECTED] http://www.postedecoute.ca/ __ Disques RSB Inc. 8400, Côte-de-Liesse Montréal, QC H4T 1G7 Téléphone : 514.342.8511 poste 228 Télécopieur : 514.342.0401 Sans-frais : 1.800.361.8153 --AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http