Re: [AOLSERVER] AOLserver, NTLM, Apache+mod_proxy,
Hi - I don't know how others feel, but one of the community issues I'm having is that a mailing list is not a great place to have 5 discussions going on at once, IMO. If the mailing list does get more active (which in a way would be great), I would probably have to just unsubscribe. I think it's great that people are having in-depth discussions about how fast-cgi works, OACS, the issues with implementing it, etc., but since that isn't on my radar and I have zero interest in OACS, it's mostly noise to me. It'd be much easier IMO if how to get fastcgi on AS were a topic on a topic-based bulletin-board. THen people could look at things that were directly relevant or of interest. The other advantage of a topic-based board is that when new people join, they can easily get a sense of history. I know (or think) there is an archived copy of the mailing list, but doubt many people are going to go through 8-10 years of a mailing list. But, they might go poking through topics that look interesting on a board. To keep people in the loop, maybe a daily digest of topics with activity could be emailed. Just a thought... Jim On 2005.02.08, John Sequeira [EMAIL PROTECTED] wrote: When you say integrated security are you talking about the NTLM auth scheme for HTTP? As long as mod_proxy properly handles HTTP Keep-Alive, -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver Applications Spec DRAFT
Maybe an interface to memcached would make sense... Nsvs are, to me, global in essence. As such, I don't really see making namespace aware as a good goal. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver facelift.
Hi - I read the entire thread mentioned below, and missed any mention of anyone saying the AS maintainers would not accept FastCGI. The other thing I guess I'm confused about, is that CGI is implemented as an AS module. Why can't someone write a FastCGI module, and whoever wants it can load it to get FCGI support? Does the core have to be modified to support FastCGI? And just as a my dog isn't in this fight observer, this discussion seems to be getting rather personal, which probably is not going to help it proceed productively. Jim Here's the kicker: The guy who'd probably do most of the work, John Sequeira, seems to think that option 1. is a better idea technically, but is leaning towards 2. instead, because after following the AOLserver multi-protocol discussions, he doubts that any FastCGI work would ever be accepted by the AOLserver maintainers, no matter what! http://openacs.org/forums/message-view?message%5fid=263944 That looks symptomatic of a problem in how the AOLserver codebase is managed to me. Dossy, doesn't it look that way to you too? -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver facelift.
I am a bit on the sidelines, because of a) time constraints, and b) we're using ns_shares heavily and so are stuck with TCL 7.x, which is no longer supported. (I know there are hacks in TCL 8 for ns_shares, but when we benchmarked them, they were much slower. So no compelling reason to upgrade, for us, at least not yet.) It seems that one of the main issues facing AS is a lack of people contributing their time to a group effort. People are doing things on their own, mainly for their own needs. It's understandable. Tweaking AS requires a high level of skill, and most of the people with that skill level have their hands full with their own projects. But it seems like having 2 separate versions will make cooperative efforts harder, not easier. JMHO. You know what I think would widen the appeal of AS? Optimize CGI and provide easy FastCGI support, or some other mechanism that makes it very fast and easy to run canned applications: bulletin board software, etc. Maybe AS CGI is already good enough to compete with Apache mod_perl, mod_php, ... I dunno. Or maybe it could never compete. But if it could, and there were tests/benchmarks to prove it, it seems that would go a long way to making AS more appealing to the general I need a good web server type of person. The recent discusion on the list about how to get an accounting application to run with mod_perl is a good example: the answer is you don't need to use mod_perl or anything like it. But I think there is confusion for people evaluating web server software, because mod_perl and mod_php are accepted as the best way of doing things - on Apache. We have an application that we've always exec'd; not through CGI, just exec. I converted it to a module because I wanted to save some state info. I expected a big performance increase too, just because it would be a module and not need to exec. But in the initial version, which was basically the same but implemented as a module instead of exec, there was hardly any performance difference. Modules are great for saving state and doing fancy stuff, but I think there is a general impression that anything good has to be done as an AS module, and that isn't true. Another thing I think would be good are some documented performance tests with well-known applications. For example, a benchmark comparing PHPBB under Apache and AS. Maybe Apache would beat the pants off AS - I dunno. Or we might be surprised. Or a whole suite of bulletin-board tests: vbulletin, phpbb, ultimatebb, etc., compared on several different web servers with the same hardware config. Published benchmarks are good because: a) they show that standard applications run b) they give people an idea of how AS compares to other options. Even if people aren't planning to run bbs software, they may be planning to run a large perl or php application, so knowing that other large applications work well gives people an extra comfort level. If it could be shown that AS runs some of these mainstream web applications better than Apache, the authors of the software would be likely to recommend AS as a solution on their site, thus increasing its popularity with little effort. Of course, if it turns out that AS performance sucked on these applications, then the goal of getting more people involved and making the server more appealing to the masses is misguided, and it would be better to focus on the smaller group of sites where deploying AS makes sense. Jim This *is* a suggestion for cooperative development of two versions in the interests of AOL and the community, perhaps with oversight by a steering committee if you like. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] The 'weird 0 length problems' revisited
You can also get a zero content length logged if you have a C module that does Ns_ConnWrite and writes the headers and content itself, then closes the connection (even if there is a valid Content-Length header on the response), or if nothing is written out - for example, your TCL script goofs up and does a premature return before doing ns_return. Jim On 2004.12.08, Bernd Eidenschink [EMAIL PROTECTED] wrote: last year there was a thread concerning a 'weird 0 length problem': Wojciech Kocjan discovered requests served with a Content-Length of 0. As I see it there was neither a solution to that problem nor a bug to be found at that time. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Yep, there are always cases where any given solution won't work well. Using the HTTP protocol to download a a 600MB CD image or to implement a movie server, is clearly different than average web usage. To display the _directory_ of the CD image, or download individual files, output spooling would be a win, IMO. For fastpath (static file serving), reading a file into a memory buffer would seem to scale well for typical web size images: 250K or less, especially considering that fastpath already uses a memory cache. I can see 3 scenarios: 1. File is small: put it in the cache, leave it there after spooling 2. File exceeds cache size limit, but is less than 1MB: put it in the cache, delete it after spooling or make it expire very soon 3. File is huge: let the connection thread spool it, like it does now, and don't spool it, or My apologies for stomping on your topic. I do think the ability to handle other protocols is a good idea, esp for sites that need to integrate user registration with a lot of different packages. Jim I don't think the read-the-whole-file-in-the-conn-thread solution scales very well when faced with a directory of CD images or a distribution of Debian packages :-) ... On Thu, 2004-08-19 at 06:14, Jim Wilcoxson wrote: I guess I was unclear in my earlier note: 1. The HTTP driver thread reads all input for request, w/o blocking. My understanding is that this is now happening in AS 4 - great! 2. A conn service thread creates the output, either by running a TCL script, executing fastpath to read a file into memory, etc. This might do stats, file I/O, database I/O, etc., and may block. The result of this thread is a memory buffer. 3. The HTTP driver thread spools the memory buffer back to the client, w/o blocking. I wasn't suggesting that there should be no blocking in step 2, when the output is being created. That's why multiple connection service threads are needed. But the problem of a connection service thread being tied up while it spools a large page/image is eliminated. In the application of an image server, it would be nice if there could be one fastpath thread that did all non-blocking filesystem I/O (actually just reading is needed), or an asynchronous sendfile, but I agree with you that there are not good portable ways to do that yet. Jim ge. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS
Hi - To put spiders in a separate thread pool, mapping requests to
thread pools by url doesn't work. You have to map by regular
expression on the User-Agent header or by IP address, assuming you can
figure out where Google's robots live. That's why I was suggesting
that the dispatching algorithm needs to be a registered proc.
Jim
The reason thread pools would be interesting to me personally is to be
able to control quality of service better. For example, I may want a
pool of 5 threads to handle search engine spidering requests, and a
pool of 5 threads to handle PHP requests, 5 threads to handle a
particular application that uses a different database, 5 threads to
handle CGI requests, 10 threads to TCL requests, 5 to handle image
serving (fastpath), 5 to handle all other requests. I'd want to
dispatch the requests to the various thread pools myself, using TCL
code. The protocol and communication method (TCP vs UDP) part are
all the same here - HTTP.
Perhaps this can all be done today with AS 4 using multiple IP
addresses - I dunno.
From sample-config.tcl ... you can map by URL, which means that by proper
partition of your URL-space you can do all you discuss above but shove
requests from spiders into their own thread pool.
#
# Example: Multiple connection thread pools.
#
# To enable:
#
# 1. Define one or more thread pools.
# 2. Configure pools as with the default server pool.
# 3. Map method/URL combinations to the pools
#
# All unmapped method/URL's will go to the default server pool.
#
#ns_section ns/server/server1/pools
#ns_section slow Slow requests here.
#ns_section fast Fast requests here.
#
#ns_section ns/server/server1/pool/slow
#ns_param map {POST /slowupload.adp}
#ns_param maxconnections 100 ;# Max connections to put on queue
#ns_param maxdropped 0 ;# Shut down if dropping too many conns
#ns_param maxthreads 20;# Tune this to scale your server
#ns_param minthreads 0 ;# Tune this to scale your server
#ns_param threadtimeout 120 ;# Idle threads die at this rate
#
#ns_section ns/server/server1/pool/fast
#ns_param map {GET /faststuff.adp}
#ns_param maxthreads 10
#
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Progress report
Hi - a few suggestions/ideas: 1. How many requests per second do you average, and what are your burst rates like? What are your vmstat numbers like during average and burst request processing, and how do they compare to when your system is lightly loaded? Getting familiar with these numbers will help you decide if you have a system capacity problem that perhaps only shows up during a traffic spike. 2. If increasing minthreads decreased the problems, it sounds like you may have trouble starting new threads during the day. Do you have a huge TCL library? This can significantly delay thread startup, because a very large TCL string gets eval'd to create your library procs. If you have a couple of threads running and a robot decides to spider your site, it may blast you with 10 requests. Your server needs to create 8 new threads, initialize them, handle these requests, and queue all new incoming requests. If a server is near its capacity, a delay of even a few seconds can take a while to recover from. Ex: your max processing rate is 10 Req/sec, your average actual request rate is 8 per second. You stop handling requests for 5 seconds, so 40 requests queue up. It'll take 20 seconds to clear this backlog, because you are near capacity. 3. How big are the pages you are returning? Do most of your pages fit into a socket buffer (usually 32K bytes)? If not, you'll tie up your connection threads spooling http output and your server will appear dead while incoming requests are queued. This is the main issue that prevents using a small maxthreads. Like others, I'd suggest you bump maxthreads up to 30 or so, but I'd also bump minthreads up to 30 because it sounds like you do have the thread startup problem I mentioned in #2. 4. I'd run a time-stamped vmstat trace throughout the day to make sure your system isn't just running out of steam. On Linux, disk I/O bottlenecks (specifically, flushing dirty buffers) can frequently cause a system to appear dead for several seconds, and recovering from this could take a while if you are already running near capacity. 5. If your system is truly idle when it is not responding to web requests, ie, vmstat shows no I/O activity and no CPU activity, you may have some kind of locking problem/bug. I don't know anything about ACS, so don't know if it extensively uses locks or not. Good luck! Jim To refresh everyone's memory, I have a situation where a couple of older ACS sites on the same system, running nsd3.3+ad13, started hanging multiple times per day. Most of the time by the time I receive notification from uptime, the problem has passed and leaves no clue behind. Sometimes I do manage to catch it still hung, but I don't know gdb well enough to know what to do, so I just restart it. Based on suggestions here, I set minthreads to 10 and threadtimeout to 3600. Maxthreads was already set to 10, so I left it alone. Since doing that there has been a significant reduction in the number of these incidents; one or less per day per site instead of every few hours. But less is not yet zero. I'm going to try experimenting with raising the above numbers some more, but anyone with a more educated guess than mine on what might be wrong would be welcome. I'd also appreciate some simple steps I could take to try to figure out what's wrong next time I catch the server in a hung state; I looked at the gdb doc Andrew posted a while back and although it would be a great reference if I knew what I was doing, it's not enough of a guide to tell me what I should be looking for. thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
maxconnections should be high, like 100. This is the number of keepalive socket connections your server maintains. These are low overhead, and make it faster for active browsers to get pages from you site since they don't have to reconnect. We use a keepalive timeout of 15 (seconds). That means roughly that 100 people can stay connected to your server at once, as long as they are making requests less than 15 seconds apart. If they go longer than that between requests, they lose their connection and have to reconnect. Jim On Aug 18, 2004, at 9:47 AM, Dossy Shiobara wrote: What class of hardware are these sites running on? maxthreads=10 is pretty low, IMHO. Running one site on a 1.2 GHz P3 running Linux, I'd comfortably set minthreads=maxthreads=30. BTW, what's the guidelines on setting maxconnections? Should it be the same as max and min threads? thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS
figure out where Google's robots live. That's why I was suggesting that the dispatching algorithm needs to be a registered proc. Perhaps it could be both. Yeah, that would be nice. No doubt many people would want to group just by URL, and that would solve several of the cases I raised. Being able to do this just with config commands would be good/easy. If combining the two methods, maybe the registered dispatcher proc only sees the unclaimed requests not already handled by config file dispatching. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
From my point of view, focusing on protocols is not necessarily the only reason to want thread pools. I'm not all that familiar with AS 4 yet, so maybe I'm being dense here... keep that in mind. :) The reason thread pools would be interesting to me personally is to be able to control quality of service better. For example, I may want a pool of 5 threads to handle search engine spidering requests, and a pool of 5 threads to handle PHP requests, 5 threads to handle a particular application that uses a different database, 5 threads to handle CGI requests, 10 threads to TCL requests, 5 to handle image serving (fastpath), 5 to handle all other requests. I'd want to dispatch the requests to the various thread pools myself, using TCL code. The protocol and communication method (TCP vs UDP) part are all the same here - HTTP. Perhaps this can all be done today with AS 4 using multiple IP addresses - I dunno. However, sometimes it would be useful to have the pool functionality with one IP address. If a search engine decides to aggressively index your site, they have to do it on the main IP address. Being able to give them a different class of service, ie, a different thread pool, would be useful. We've already done a similar thing, by setting up multiple AS instances. But it would have been easier in many cases to set it up with thread pools. Much less code on our end to change. A way to accomplish the same thing today is by putting an intelligent switch/cache/proxy device in front of multiple servers, and having it route certain kinds of requests to certain servers. The problem from my point of view is: - the switch is another device to manage - it's an extra $5000+ - it's often a proprietary solution - if you look at client IP addresses, you have IP address issues to deal with, or have to set up some fancy routing, or have to modify your code to get the true peer address from a made-up header - you have to learn how to use the device's routing rules instead of making rules yourself, with TCL - hit logging will probably work differently; another difference to handle Another advantage to using thread pools in this application is that it may help a site be less vulnerable to DOS attacks. If a site is running 3rd party bulletin board software for example, and someone decides to run an attack against it, it may still kill that the board service/thread pool, but the other services running on the same server instance would not be unaffected. Without this partitioning, the attack might use all of the threads and kill the site. Maybe the thread pools could also have priorities too. So I could make dispatching decisions based on the current request load, like, If there are more than 5 threads active in the main pool, hold off executing any new requests in the bulletin board pool, ie, queue the request, but don't let it execute. In an ideal world, there would always be enough resource to do everything, but in the real world, there are sometimes momentary resource shortages that cause things to get slow. If something is going to be slow, I'd rather it be a non-essential service like a bulletin board rather than the threads generating our main pages. Is this something critical I can't live without? No. But it seems useful. Jim I definitely vote for a). I think AS has a very powerful framework that can be used outside of http. --- Zoran Vasiljevic [EMAIL PROTECTED] wrote: On Tuesday 17 August 2004 10:32, Zoran Vasiljevic wrote: Hi! I would like to start a discussion which will eventually lead to one of the two proposals listed below (one from Vlad Seryakov and the other from Stephen Deasey) included in the core-server distribution. Eh... wrong turn again... By start a discussion, I really expected the people to respond with: a. YES, I like AS to be more powerful multiprotocol server instead of being a powerful http-server only. And, I like proposal X better than Y because of Z. b. NO, I do not need AS doing anything else than http and am perfectly happy with status-quo now. Just forget the whole thing. I apologize for not being more precise. If a. turns out to be everbody's darling, then there are two excellent proposals for making this work. If somebody has a better solution, lets see this as RFE. Then, get a couple of (knowledgeable) people vote in democratic way about what's the better solution. I think there is a AOLSERVER-DEVEL (or such) list where deeper technical issues can be discussed in a smaller audience. If b. well, than it is clear. We just leave it as-is. Can we agree on this? Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver -
Re: [AOLSERVER] file upload corrupts jpg images
I seem to recall that the form file upload handler was rewritten in C for 3.5. It was in TCL before that. You could write a test script that does your file upload and runs an md5 on the temp file, while it still exists, with the exec command. Do a bunch of those with the old server and new server to see if there are any differences in the checksums. That will give you a test case and then you can start fiddling with things to find out why they are different, assuming that is the problem. Jim So, it appears that the file content is being rearranged. (on file upload w/3.5) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] SQLite
A suggestion: the SQLite DB driver could keep a serial number that is incremented whenever the DB is modified by the driver. The driver could then re-open the DB if the file handle it gets from gethandle is stale. It wouldn't work in the case of another process outside the web server modifying the database, but maybe that could be done with a periodic check of the DB file's timestamp. On Wed, Jul 14, 2004 at 02:51:44PM -0400, Dossy wrote: SQLite version 3.0 is made possible in part by AOL developers supporting and embracing great Open-Source Software. I wonder what this is all about. Anyone know? No idea. Sounds interesting, though. :) And, from the FAQ at http://www.sqlite.org/faq.html: | Note that if two or more threads have the same database open and | one thread creates a new table or index, the other threads might | not be able to see the new table right away. You might have to get | the other threads to close and reopen their connection to the | database before they will be able to see the new table. Pargraph #3 bothers me. If this is the case (changes in one thread not picked up by the other threads unless they close/reopen the DB), that's a pretty tough limitation. Dossy, like it or not, that's because by design, the ONLY operating system services SQLite uses for locking are a file locks on the database file. (More or less; if I remember right, it's actually a bit more complicated than that, but that's the general idea.) For AOLserver use, you basically have two simple options: 1. Open and close the SQLite db file for EVERY transaction. This is the normal way to use SQLite, and is definitely what you have to do if other processes might also be accessing the same database. 2. Keep the database open, and serialize all SQLite operations through a single thread. I would prefer to see an optional single-process many-threads MVCC concurrency model for SQLite, and several variants of that have been discussed rather extensively on the SQLite list. But so far no one has implemented it, and it's rather outside Dr. Hipp's primary design goals for SQLite. SQLite 3.0 introdues some new stuff to improve concurrency, but it doesn't go nearly as far as full-blown MVCC. That said, the SQLite wiki has numbers suggesting that the concurrency achievable even with the older 2.8.x SQLite is probably adequate for many small to medium websites: http://www.sqlite.org/cvstrac/wiki?p=WhenToUseSqlite My personal SWAG, is that if you can run your website ok on MySQL, you can probably run it just fine on SQLite too. -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Virtual hosting under 4.x and default Host: header
How do people feel about this? Is glob matching wanted enough to incur that per-request cost -- only if you're configured for software virtual hosting, of course. How about: try hash match if not found try glob matching against the list of virt. server wildcards if found a match use that server else use default server You only pay the glob penalty if the admin configures wildcard server names. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TclX keyed list changes in HEAD
Hi - here are my thoughts on compatibility: Breaking compatibility at the TCL level is worse than breaking it at the C level. If someone wants to write a new, incompatible version of ns_return for example, it should be called something else and a compatibility wrapper written for ns_return that calls the new thing. In the best case, TCL compatibility should NEVER be broken. If we end up with a huge compatbility library, perhaps all written in TCL, that's much preferable to telling people to change their code. Breaking compatibility at the C level is more acceptable to me, because my guess is that very few people are using it. We have a small handful of C modules, so yeah, we may have to change them if the C API changes, but it would not be a big deal. Even if incompatible changes are allowed at major versions, what may very well happen is that it fractures the community and leaves some people running old versions longer than they want. We're in this situation now because of ns_shares: someone decided it was inconvenient to provide ns_share because of TCL core modifications, which I totally understand. But, we have a code base with over 800 ns_share statements, and probably 5x that many variable references. It will be a cold day in Hell before we rewrite all our code to use nsv arrays, just because someone decided that ns_shares are inconvenient to support. If ns_share ever goes away completely, we won't be upgrading, no matter how great a new version is. The other paradox with good, reliable software is that people don't have as much of a pressing need to upgrade it. So while a development group is releasing version after version of a product, and may have even released several production versions, there will be many customers that have not even tried the new version. Eventually, there will be a pressing need that forces the customer to consider upgrading. The Linux OS is an example of this, for us. We didn't upgrade to 2.4 until November of 2003, even though 2.4 had been released for a long time, because 2.2 was working fine for our needs. The 2 things that forced us to upgrade were that we needed to access more main memory, and our storage requirements increased to the point that we couldn't afford to do fsck's and needed a journaling file system. My point is, production customers may be running old versions much longer than developers think. If a customer has been happily running 3.4 AS for a few years and a compelling reason comes along for them to upgrade, and the current version of AS is 6.5, they don't want to run into a big incompatibility hurdle. If they run into too big of a hurdle, they will likely start considering all options, including moving to a different platform that they perceive is more stable from a compatibility point of view. Jim Zoran implemented some recent changes to TclX keyed lists in CVS HEAD which changed the C API, which broke at least some code at AOL which was still using the old C API. Rather than a wholesale back-out of the changes, I'd like to discuss two things: 1) Should changes that break backwards compatibility be allowed between minor revisions (i.e., 4.0 - 4.1) or should they be limited to major releases only (i.e., 4.x - 5.x). 2) Can we quickly implement some backwards compatibility for the TclX keyed list C API so that existing C code won't need to be modified/updated to use Zoran's new C API? What's the best way to do this? Can it be done through #define's? Or thin wrapper C procs that call the new C procs? Or, can we simply rename Zoran's new C procs to the old names to preserve compatibility? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ANN: Searchable list archive at Gmane.org
Hi Dossy - sorry, your email was filtered. Because I use a published email address, I had to take the sledgehammer approach to spam filtering and toss anything that comes directly from a dsl or cable connected computer. Sad, but necessary for over 500 spams per day. The files I have are in listserve dump format, like this: = Date: Sat, 23 Mar 1996 16:37:24 -0500 Reply-To: Web Development with GNNserver [EMAIL PROTECTED] Sender: Web Development with GNNserver [EMAIL PROTECTED] From: Jeff Rawlings [EMAIL PROTECTED] March 23, 1996 Dear NaviSoft Customer: NaviSoft has always offered our customers the best web products and services available. Over the past year we have continued to refine our features and enhance our customer support. In October, we announced the launch of NaviService, which integrated both NaviPress and NaviServer into a powerful web hosting service. We are now integrating our web hosting service into GNN (Global Network Navigator), America Online's flagship Internet Service Provider, and we announce four exciting new developments: = Date: Sat, 23 Mar 1996 16:37:24 -0500 ... So I think I can reformat it to mbox w/o too much trouble. I will send you one file as a test. The archive is in 41 chunks. And I'll see about adding a whitelist to our spam filter! Jim On 2004.06.18, Andrew Piskorski [EMAIL PROTECTED] wrote: On Fri, Jun 11, 2004 at 02:50:40PM -0700, Jim Wilcoxson wrote: If you are interested in adding historical stuff to the archive, I have the mailing list from March 23rd, 1996 to Oct 9, 1999. Lemme know. That would be neat, please do add those really old archives. I replied to Jim directly off-list indicating that I was interested, but I need to know what format his archive of the mailing list is in. The Gmane.org folks need it in Unix mbox format. Jim, what format do you have the messages in? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_set questions
Sets allow entries with duplicate keys. If duplicates are returned in
the list for this new ns_set keys, that's fine, except that a naiive
programmer can't go through the list of keys returned and fetch the
corresponding values. You'd have to do a foreach loop on the keys,
increment a counter, and use the counter to lindex into the ns_set values
result:
set values [ns_set values $myset]
set i -1
foreach key [ns_set keys $myset] {
incr i ;# put it here so that continues can be used to exit early
set val [lindex $values $i]
do whatever
}
It seems to me that this is worse than the current way of using a set
index with ns_set key and ns_set value:
set size [ns_set size $myset]
for {set i 0} {$i $size} {incr i} {
set key [ns_set key $myset $i]
set val [ns_set value $myset $i]
do whatever
}
IMO, a better approach (or another one at least) would be to return a
list of key value pairs with something like ns_set list, ie:
{{key1 val1} {key2 val2} ...}
Then a loop like this can be used:
foreach setel [ns_set list $myset] {
set key [lindex $setel 0]
set val [lindex $setel 1]
do whatever
}
Another option would be ns_set aget to return alternating keys and
values in a list that could be used as input to array set - sort of,
ignoring the multiple key issue - to convert sets to arrays, and
ns_set aset could be used to convert from arrays to sets, like:
set myset [ns_set create]
ns_set aset $myset [array get myarray]
I don't know how often these would be used in practice though, because
I think experienced TCL programmers quickly avoid using ns_sets except
where they have to, like for interfacing with smallish nsd data
structures.
Jim
On Fri, Jun 11, 2004 at 02:31:18AM -0400, Jamie Rasmussen wrote:
While adding documentation to the Wiki for ns_set (see
http://panoptic.com/wiki/aolserver/ns_set) I noticed that ns_set has ns_set
Also, I think it would be useful to add some new subcommands, ns_set keys
$setId and ns_set values $setId. These would return a list of the keys or
values in the ns_set respectively. While you can get these lists from
looping with ns_set key / value / array, it is convenient to have them as
Sounds like a good idea to me. Having to manually iterate over the
ns_set to find all matching keys always did seem awfully strange to
me.
What do people think? I'd be happy to submit a patch for the changes. Are
there any other ns_set modifications you've been wanting?
About two years ago, I thought I wanted some, so I patched the stuff I
wanted into my local AOLserver code. Ah, it was the findall and
findmatch stuff:
http://sourceforge.net/tracker/?func=detailaid=530808group_id=3152atid=303152
But then I learned what horrible performance ns_set has for large
quantities of data [I just added a note to the Wiki about that now],
which was exactly the situation I was using findall and findmatch
with. So I switched to Tcl Arrays, my need for a richer ns_set API
evaporated then and there, and I never did post my cleaned up patch.
(I still have it in my CVS though.)
--
Andrew Piskorski [EMAIL PROTECTED]
http://www.piskorski.com/
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] ANN: Searchable list archive at Gmane.org
If you are interested in adding historical stuff to the archive, I have the mailing list from March 23rd, 1996 to Oct 9, 1999. Lemme know. Jim As an added bonus (!!!), I've supplied my own personal mail archive of all the messages I've received from the list since January 2000 when the list was created, and the kind folks at Gmane have started importing them all! So, very soon you will be able to search the archive all the way back to the beginning. (I think my archive may be missing a total of 50 messages over the past 4 years. Oh well, I'm sorry about that.) I hope this new archive will be useful to folks, and hoepfully will help more people once the mailing list contents can be found via Google. I'll continue to gather everyone's feedback and try to come up with other common themes, but the next big one is, of course, the documentation which is being actively worked on. Again, I plead with everyone: help out, pitch in and contribute to the documentation! Either by helping to write it, or by making specific requests for what documentation you feel would be most useful! As usual, feel free to send requests directly to me off-list and I'll take care of the rest. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_conn without passing it around
For 3.4, it's:
Ns_Conn *conn;
/* get connection structure */
conn = Ns_TclGetConn(interp);
if (conn == NULL) {
Tcl_AppendResult(interp, NULL conn??, NULL);
return TCL_ERROR;
}
Jim
Is there a way to obtain the current connection's Ns_conn if
it's not passed into a C function explicitely? Is it accessible
somewhere globally or via a function at the C level?
Thanks for any help.
--
-- Mike
Mike Schilli
[EMAIL PROTECTED]
AOL Host Infrastructure Engineering
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] ADP Processing happens when...?
I made this suggestion a while ago (sending the output in an output thread). One argument against it at the time is that on a multi-processor machine, you may not be able to take advantage of all the processors. However, input and output spooling is not CPU intensive, so I don't think it's all that necessary to have multiple input and output threads running. As far as memory considerations, the same amount of memory is going to be needed whether there are N threads, each spooling 1 output request, or 1 output process spooling all N output requests. An alternative is to configure your socket output buffer size so that the majority of your output responses will fit. You'll get the same basic effect as single-process output spooling, but your OS will do all of the work. Jim That got me to thinking about how AOLserver works on responses - So a single thread handles the receiving of the request - which makes perfect sense on requests around 32K or so, which is probably 99.5% of them. What about on output? In other words, a request comes in, the single IO thread reads it, then once its finished it gets handed out to a thread for processing. What happens when its done? Does that thread do all the output on its own? In my imaginary world, I would love it if the thread handed the output data back to the central I/O thread, which could return it to the client at its own pace. AOLserver's I/O thread would get huge in terms of memory for doing this, but it _should_ act as a powerful connection multiplexor - you can imagine possibly 2x maxthreads connecting, and if the individual threads finished off the connections quickly, returning data to the central thread - well, it would pretty much obviate the need for caching proxy servers in front of AOLserver, right? If it already works this way - then architectural bits like that should be showcased so that more users choose AOLserver. If it doesn't - does what I say make sense? Would it have too much memory impact? Sorry for the ramble, your previous response and that URL got me to thinking about an idea I had had before... On Tue, 11 May 2004 15:35:40 -0400, Nathan Folkman [EMAIL PROTECTED] wrote: Exactly. The idea is to keep I/O events isolated to the main driver thread, which in theory should be more efficient. The goal is to not tie up connection threads, and have them sitting idle waiting for I/O. This model is kind of the best of both worlds - single-threaded event loop that accepts connections and consumes the request, and multi-threaded connection threads to do the actual work of the request. The ideas are similar to work that's discussed here: http://www.eecs.harvard.edu/~mdw/proj/seda/ At least this is how I remember things... ;-) - Nathan Tom Jackson wrote on 5/11/2004, 3:23 PM: You have to read all the data anyway, but now the POSTed data is read in so you don't have a tcl procedure slowly reading data and controlling the connection. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. A -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] SQL error reporting
Yes, it's for catch.
A useful development hack is to change nsadmin/modules/tcl/file.tcl so
that when an error occurs, the error info is passed to an error handler
and displayed as a web page. That way the developer doesn't have to
keep a tail running of the server log when testing and paw through it.
But the detailed error info was being wiped out by the not a statement
returning rows change. This change restores the detailed error info.
Jim
Is this only for the catch info in Tcl? In the server log it is still
reported with the orriginal server message, one error before the not
returning rows one.
Bas.
Jim Wilcoxson said:
In one of the 3.x releases, I think after 3.2, AS stopped reporting
the details of SQL select errors. Instead of giving the detailed
SQL error, like expected comma after blah, AS started reporting
Query was not a statement returning rows. This isn't very useful
for error trapping and reporting.
Here's a patch to go back to the old behavior. Not sure whether this
even applies to 4x.
*** dbdrv.c.~1~ Tue Jan 16 10:14:27 2001
--- dbdrv.c Tue Dec 9 10:34:35 2003
***
*** 367,368
--- 367,369
Ns_Set *setPtr = NULL;
+ int status = NS_ERROR;
***
*** 371,377
if (driverPtr-execProc != NULL) {
! if (Ns_DbExec(handle, sql) == NS_ROWS) {
! setPtr = Ns_DbBindRow(handle);
! } else {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
}
--- 372,379
if (driverPtr-execProc != NULL) {
! status = Ns_DbExec(handle, sql);
! if (status == NS_DML) {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
+ } else if (status == NS_ROWS) {
+ setPtr = Ns_DbBindRow(handle);
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Bug #879076 (Output header issue)
Hi - For 3.4x, look in nsd/return.c and nsd/fastpath.c; fastpath is calling Ns_ConnSetLastModifiedHeader in return.c to set Last-Modified. This routine calls Ns_ConnSetHeaders, which does an ns_set put, but it could easily call Ns_ConnCondSetHeaders, which would only do the put if the header doesn't exist. Changing the put to an update would not be a good idea IMO, because then the result would depend on whether the TCL was executed before or after fastpath. Ns_ConnCondSetHeaders seems like the better approach. The same thing could be done for Content-Type (mentioned in the online bug report). Are you executing your TCL stuff in a filter/trace? Or doing an ns_returnfile? Those are the only ways I can see where you would be executing TCL to affect the headers but still ending up in the fastpath code. I've had a similar problem in the past: I needed to do an ns_returnfile, but didn't want the Last-Modified header to be sent out at all because I didn't want the result cached. Never figured out how to do that. Good luck, Jim Hello good people, A friend has helped me comment on an AOLserver bug, but I was hoping to also seek assistance here. For reference, this is the bug: http://sourceforge.net/tracker/index.php?func=detailaid=879076group_id=3152atid=103152 To answer Dossy's question, I have tried: ns_set update [ns_conn outputheaders] key value This however still results in a duplicate header (in my case Content-Type). I have confirmed this bug in both AOLserver 3.3.1 and 3.4.2. Restating what I said in the bug report: Is there any way to workaround this from within the Tcl/ADP page? If not, does anyone know which function call this may relate to within the C API? I'd be happy to submit a patch if no one else has the time to fix it. Also, does anyone know if this has been fixed in AOLserver 4? If so, would the fix be easy to back port to 3.3.1 or 3.3.4? Thank you very much, Andrew -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] SQL error reporting
In one of the 3.x releases, I think after 3.2, AS stopped reporting
the details of SQL select errors. Instead of giving the detailed
SQL error, like expected comma after blah, AS started reporting
Query was not a statement returning rows. This isn't very useful
for error trapping and reporting.
Here's a patch to go back to the old behavior. Not sure whether this
even applies to 4x.
*** dbdrv.c.~1~ Tue Jan 16 10:14:27 2001
--- dbdrv.c Tue Dec 9 10:34:35 2003
***
*** 367,368
--- 367,369
Ns_Set *setPtr = NULL;
+ int status = NS_ERROR;
***
*** 371,377
if (driverPtr-execProc != NULL) {
! if (Ns_DbExec(handle, sql) == NS_ROWS) {
! setPtr = Ns_DbBindRow(handle);
! } else {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
}
--- 372,379
if (driverPtr-execProc != NULL) {
! status = Ns_DbExec(handle, sql);
! if (status == NS_DML) {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
+ } else if (status == NS_ROWS) {
+ setPtr = Ns_DbBindRow(handle);
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] aolserver 3.4.2 configuration questions
Your servername may not be set correctly in nsd.tcl, so it isn't
looking at the enabletcl directive in your /ns/server/servername
section.
Jim
It is very strange but I don't see no 'Notice: tcl:
enabling .tcl pages'...
And it also seems that my enabbletcl true statement in
my config file has no influence!!
I can change that string to everything I want..the
server just starts and doesn't complain about even
wrong things I enter there...
Still not a clue as to what is happening. My log file
contains nothing about it.
--- Mark Page [EMAIL PROTECTED] wrote:
On the running .tcl pages part (there are others who
can comment better
on your postgres questions);
Try looking at what shows up in the server log file.
This file will
either be flowing to standard-out, or to the
configured 'serverlog' path
(defaults to $homedir/log/server.log; you've
specified homedir ==
/home/rgilaard/).
You should see a 'Notice: tcl: enabling .tcl pages'
message near the
start confirming that .tcl pages have been correctly
enabled.
Also, there may be something else going awry, which
could be identified
by entries in this log, perhaps associated with
trying to service the
.tcl page request.
Dino Vliet wrote on 1/20/2004, 11:00 AM:
Dear all,
I've installed aoserver 3.4.2 on my freebsd 4.9
laptop
for developing and testing some web applications
I'm
planning to create. The installation went ok and
I can
serve static pages. However, when I try to feed
my
system some .tcl pages under pageroot I get the
whole
page as text and not the results of the tcl
statements
I wrote.
For instance, this little hello world tcl program
doesn't appear as expected in the browser.
hello.tcl
ns_return 200 text/html html
head
titleTesting/title
/head
body
pHello, world/p
/body
/html
The output of this when I put the commands in a
.tcl
file under pageroot is:
ns_return 200 text/html html
head
titleTesting/title
/head
body
pHello, world/p
/body
/html
I then tried something similar but the results
were
the same:
ns_write HTTP/1.0 200 OK
MIME-Version: 1.0
Content-Type: text/html
ns_write html
head
titleTesting/title
/head
body
pHello, world/p
/body
/html
I also have a question regarding database access
with
the postgres driver. I have installed postgres
7.3.4
and it works fine. However, I wasn't able to find
no
postgres.so file in any directory on my system. I
thought aolserver would install it somewhere and
I
could use that location to find it. What I did
find
was a /usr/local/lib/postgresql/plpgsql.so file,
and I
added it to db section, but I don't think this
will do
the trick:-)
Here is my nsd configuration file:
# $Header:
/cvsroot/aolserver/aolserver/nsd/sample-config.tcl,v
1.2 2000/12/13 21:50:27 kriston Exp $
ns_log notice config.tcl: starting to read
config
file...
#
# Set some Tcl variables that are commonly used
throughout this file.
#
set httpport 8000
set httpsport 8443
# The hostname and address should be set to
actual
values.
set hostname localhost
set address127.0.0.1
set servername server1
set serverdesc Server Name
set homedir/home/rgilaard/
set bindir
/usr/local/aolserver/bin/
set pageroot
${homedir}/servers/${servername}/pages/
set directoryfile
index.adp,index.html,index.htm
set ext [info sharedlibextension]
# nsssl: Only loads if keyfile.pem and
certfile.pem
exist.
#set sslmodule nsssl${ext} ;#
Domestic
128-bit/1024-bit SSL.
set sslmodule nsssle${ext} ;#
Exportable
40-bit/512-bit SSL.
set sslkeyfile
${homedir}/servers/${servername}/modules/nsssl/keyfile.pem
set sslcertfile
${homedir}/servers/${servername}/modules/nsssl/certfile.pem
# nscp: Uncomment the sample password and log in
with
nsadmin, password x,
# type ns_crypt newpassword salt and put
the
encrypted string below.
set nscp_port
set nscp_addr 127.0.0.1
set nscp_user
#set nscp_user nsadmin:t2GqvvaiIUbF2: ;# sample
user=nsadmin, pw=x.
#
# Global server parameters
#
ns_section ns/parameters
ns_param home$homedir
ns_param debug false
#
# Thread library (nsthread) parameters
#
ns_section ns/threads
ns_param mutexmeter true ;# measure
lock
contention
#ns_param stacksize [expr 128*1024] ;#
Per-thread
stack size.
#
# MIME
Re: [AOLSERVER] nssock: server ready - resuming
These messages mean that the server received a request, but there were no free connection threads to pass the request to, ie, all of your threads were busy. The naive solution is to increase MaxThreads, but this likely won't solve your problem. For example, if your machine is out of CPU, setting MaxThreads higher will only cause more processes to be waiting for CPU, and eventually you'll get this error again. Same thing if you have used all your IO capacity on the machine. If you have a special case, where your server threads are coordinating with other processes and it takes a long time, doing sleeps, or something else where the threads are waiting a lot and not consuming resources, then bumping MaxThreads might help. Jim Hi ! I'm using AOLserver/3.3.1+ad13 and it is very slow. We thought it was slow due to OACS4 that runs on it, but now I noticed the log is full with : [20/Jan/2004:01:28:58][5962.4101][-conn1-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.6151][-conn3-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.5126][-conn2-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.4101][-conn1-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.3076][-conn0-] Notice: nssock: server ready - resuming I counted 2793 appearences in a log of 6 days. The server is listening on port 8000 and is accessed by an apache server VirtualHost doing ProxyPass. (if that has anything to do with it). The server is quite busy - running 3 instances of ACS on AOLserver, some apache websites, zope, qmail, ftp etc. BUT the other 2 sites running on AOLserver are not slow. Does anybody have a tip or a clue about these nssock msgs? Thanks, Yuval. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
Here's a script to dump out the memory pool info:
set text
set elname blocksize nfree nget nput nrequest nlock nwait
foreach pel [ns_info pools] {
append text \nPoolname: [lindex $pel 0]\n
set i 0
foreach el [lindex $pel 1] {
append text [lindex $elname $i]: $el\n
incr i
}
}
ns_return 200 text/plain $text
return
It seems that one way to address this memory/heap problem is to force
threads to exit after a certain amount of time idle, freeing their
entire heap area. There is code in AS to do that, but I've never
gotten it to work successfully because AS tends to round-robin the
workload to all threads. So they never idle.
If threads went to the top of a work queue when they were fininshed
instead of the bottom, and threads were always removed from the top,
the idle thread exit code would stand a chance of working. Also, a
system over-configured for 30 threads would automatically adjust
itself to use only 5-6 threads, or whatever the actual load required;
5-6 fragmented heaps is better than 30.
Jim
In a message dated 1/10/04 12:03:58 AM, [EMAIL PROTECTED] writes:
A dynamic content server tends to grow in our
experience, and I've always assumed that's because of heap
fragmentation problems - not necessarily leaks.=A0 Just restart your
server once a day/week.
=20
Yes -- memory fragementation is a problem in AOLserver which does lead to=20
large processes. Actually, it's more accurate to say we accept fragmenta=
tion=20
as a side effect of Tcl's threaded memory allocator which is a per-thread,=20
power-of-2 alloctor. This allocator essentially eliminates deadly lock=20
contention problems of most normal allocators but requires high overhead and=
leads to=20
fragmentation. You can peak into the current state of the allocator with t=
he=20
ns_info pools command.
It's possible to compile Tcl without the threaded allocator by editing Tcl's=
=20
Makefile to remove the -DUSE_THREAD_ALLOC=3D1. Then you could try alternat=
e=20
malloc packages which may provide better behavior for you app., i.e.:
normal config:
ns_malloc - Tcl_Alloc - Tcl threaded allocator -- malloc/Tcl_Alloc not=20
compatible
disabled config:
ns_malloc - Tcl_Alloc - malloc -- you can pre-load some other malloc=20
shared library
HORD is often cited as a good alternatiive: =20
http://www.cs.umass.edu/~emery/hoard/. =20
-Jim
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Mention of AOLserver in Feb 2004 Linux Journal.
Mathopd will have higher latency as an image server, because it is single threaded and doesn't do asynchronous disk I/O. So on a high-end server, the number of IOs/sec it can generate is limited. AS does not have this problem since it is multi-threaded. However, mathopd services requests in a very fair manner: in an ab test, the difference between minimum request time and maximum request time is very small. On AS, this is not the case, and there are relatively large variances in request service time, indicating that some threads are being starved. This could be an AS design issue or an OS scheduling/resource issue - dunno. Mathopd is definitely the faster engine, but its design limits its capacity. Mathopd with asynchronous disk I/O would be awesome. Mathopd: Concurrency Level: 15 Time taken for tests: 0.976 seconds Complete requests: 1 Failed requests:0 Total transferred: 2100210 bytes HTML transferred: 450045 bytes Requests per second:10245.90 Transfer rate: 2151.85 kb/s received Connnection Times (ms) min avg max Connect:0 0 1 Processing: 0 0 1 Total: 0 0 2 AOLServer 3.4: - Concurrency Level: 15 Time taken for tests: 1.822 seconds Complete requests: 1 Failed requests:0 Total transferred: 255 bytes HTML transferred: 45 bytes Requests per second:5488.47 Transfer rate: 1399.56 kb/s received Connnection Times (ms) min avg max Connect:0 0 2 Processing: 0 1 210 Total: 0 1 212 On Thu, Jan 08, 2004 at 09:15:49PM -0500, Lamar Owen wrote: There is a good, if inconspicuous, mention of AOLserver in the Feb LJ. On page 46, in the feature on the Magnatune record label, the statement is made: Apache 2 running PHP and OpenSSL serves all the HTML pages. When Magnatune was Slashdotted, I found that Apache could not keep up with the load for images. All HTTP image requests now are off-loaded to AOLserver, which had the lowest latency to serve images at high speeds. Later: Mathopd [which they use to serve the very large streaming audio files] has more latency than AOLserver, which is why we don't use it to serve small images. Why would anyone care about the latency of serving small images? Last I heard a human being viewing images in a browser is not exactly senstive to small latencies the way a parellel MPI program might be, after all. Or are they talking about absurdly large latency differences between Mathopd and AOLserver, like several seconds or more? -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
My personal opinion is that one of the responses to this simple question was pretty shitty, and totally inaccurate. In our experience, an image-serving copy of AOLserver with 30 threads configured runs in about 25MB, regardless of the load, and serving dynamic content on 30 threads takes 100-200MB, depending on the amount of caching you're doing with ns_shares and other types of persistent in-memory data storage, and the size of your preloaded tcl modules library. Of course, if you configure your fastpath cache to be huge or make some other drastic config change in nsd.tcl, it will show up as increased process size. 30 threads is a lot; most sites won't need that, but it all depends on the size of your OS's socket buffers, average size of the content you're serving, etc. An image-serving AOLserver, with TCL disabled, tends to be very stable memory-wise. A dynamic content server tends to grow in our experience, and I've always assumed that's because of heap fragmentation problems - not necessarily leaks. Just restart your server once a day/week. Jim --- John Shafto [EMAIL PROTECTED] wrote: Doing what? This particular machine only has a 512mb of ram, and serves mostly static content. I have bigger plans for it though. AOLserver is NOT Apache. Get used to that. I'll try to keep them straight. Thanks for the tip. Am I the only one who feels somewhat uncomfortable by this response? If I've only got 50mb of static content that it's serving up, and then pushing some db stuff through the back, what on earth would possibly make the process use 2gb+? I'd be getting worried then as well... = -- live- http://www.thedenofsin.org/ to- AIM: IMFDUP _-jupiter accepts your offer-_ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] how to return a custom 404 page with code 404?
Instead of putting notfound.html as your parameter, you could use
notfound.tcl and create a new tcl script with this line:
ns_returnfile 404 text/html [ns_server pageroot/global/notfound.html
Save this script as /global/notfound.tcl
Good luck!
Jim
I would like my server to return a custom html page (or even better a
tcl/adp pair) when a url is not found, with an http status code of
404.
When setting this in the config file:
ns_section ns/server/${servername}/redirects
ns_param 404 /global/notfound.html ;# Not Found error page
it does return my notfound.html page, but with a 200 status code.
I know I can do 'ns_return 404 text/html' manually somewhere, but
where would be the best place to call that? In a custom notfound
tcl/adp pair? In a filter? Or is there a cleaner way to achieve what I
want?
I only tested this on OpenACS by the way, which calls
ns_returnnotfound I believe.
til
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] how to return a custom 404 page with code 404?
And don't forget the closing bracket after pageroot, like I did! ;) Instead of putting notfound.html as your parameter, you could use notfound.tcl and create a new tcl script with this line: ns_returnfile 404 text/html [ns_server pageroot/global/notfound.html Save this script as /global/notfound.tcl Good luck! Jim I would like my server to return a custom html page (or even better a tcl/adp pair) when a url is not found, with an http status code of 404. ... -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] how to return a custom 404 page with code 404?
Because this is implemented using a general internal redirect mechanism in the server. For AS to return the 404 automagically, it would need a default http status field in the connection structure to keep track of this state, and then there would be some confusion over what to do with the default status vs explicit status when someone did: ns_returnfile 200 text/html blah which is what's is happening when your static html file is returned. I do agree that it's kinda weird/bogus that a code 200 gets sent out for static files used as the not found file, but it seems more of a documentation/how to workaround issue to me rather than needing a server hack. Another thing your TCL 404 handler could do is act as a forwarding service. For example, a friendly ISP might let a customer register a forwarding web address for a while if they leave. When a 404 occurs, the ISP looks in the forwarding table and returns a 301 - Moved http status rather than the 404. Wouldn't be possible if the server was hacked to always return 404... Jim I was just wondering if there was a more obvious way to accomplish that - why doesn't aolserver return 404 status by default when having custom error pages? Is there a reason for this or is it just oversight? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] HTTP_RAW_POST_DATA equivalent in AOLSERVER TCL
Try ns_writecontent. That will write the post data to a file (I think) and then you can muck with the file data using TCL. So, for example: # get interp # for a unique file name (or use ns_mktemp, or ...) set me [ns_info interp] set fd [open /tmp/post$me w+] ns_unlink /tmp/post$me ns_writecontent $fd seek $fd 0 set postdata [read $fd] close $fd Use this instead of ns_getform or ns_form. It'll probably have TCL issues if there are nulls in the post data. Good luck! Jim Hello, I would like to ask if there is an equivalent of PHP's $HTTP_RAW_POST_DATA global variable in AOLSERVER using TCL. $HTTP_RAW_POST_DATA gets an entire form post in raw format. This allows a great deal of flexibility as developers like me can parse the data from an unconventional post (e.g. multi-mime multi-part). I tried ns_getform but it doesn't seem to work because it puts form posts into an ns_set. Thanks, Hamilton -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Jim's Sunday commits and AOLserver 4.1
In a message dated 11/18/2003 12:03:27 AM Eastern Standard Time, [EMAIL PROTECTED] writes: Subject: Re: Jim's Sunday commits and AOLserver 4.1 On 2003.11.17, Jim Davidson [EMAIL PROTECTED] wrote: Glad somebody noticed the new code :) The changes to kick of 4.1 were in two areas: 1. Adding an I/O event callback interface for connections before they are queued for processing (called pre-queue). The interface is a bit tricky to use and there's no documentation or example module (yet). The general idea is to enable efficient pre-fetch of remote resources prior to running the connection, keeping the connection threads from stalling on I/O. For the ordinary nssock module, there should be no impact. For the SSL module, reads are now handled by special reader threads, not the connection threads. This may require moving some state from thread to connection and/or socket local storage. For requests that include file uploads, does a single, event-driven thread read in the entire request now, including the file data, before starting a connection thread? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd binding to 1 port?
You can easily bind to more than one port/IP address, however, I
believe all requests in 3.x are dumped into a common connection queue,
which does not solve the original problem. It sounds like you want
multiple connection queues, so that one can be overloaded yet the
other is still getting serviced regularly. That doesn't seem possible
in 3.x to my knowledge, at least not without hacking.
Jim
You definitely can do this in AOLserver 4 by loading more than one nssock
module.
In AOLserver 4 if you are using the built in 'virtual server' support, you
configure one global nssock and it listens on whatver port you configure it
to.
You can then have an individual virtual server (within the same nsd process)
listen to a different port by giving it its own server specific nssock
module.
Whether this is possible with AOLserver 3.x I don't know.
Try looking in the list archives
You could try something like this to get it to listen on port 8080:
ns_section ns/server/$servername/module/nssock8080
ns_paramhostname$hostname
;# This is not the same as your hostname
ns_paramaddress $address
;# This is not the same as your host addr
ns_parambufsize [expr 16 * 1024]
;# IO buffer size
ns_paramrcvbuf 0
;# Use OS defaults
ns_paramsndbuf 0
;# Use OS defaults
ns_paramsocktimeout 30
;# Connection timeout
ns_paramsendwait30
;# Timeout for sending of data
ns_paramrecvwait30
;# Timeout for receiving data
ns_paramclosewait 2
;# Timeout for closing socket
ns_paramkeepwait30
;# Keepalive inactivity timeout
ns_parambacklog 5
;# 5 pending connections
ns_paramport8080
;# Port for HTTP (typically 80)
ns_paramlocation
;# URL for auto-redirects (trailing slash)
ns_section ns/server/${servername}/modules
ns_paramnssock8080 ${bindir}/nssock${ext}
-Original Message-
From: AOLserver Discussion
[mailto:[EMAIL PROTECTED] Behalf
Of Scott Laplante
Sent: Monday, November 03, 2003 9:36 PM
To: [EMAIL PROTECTED]
Subject: [AOLSERVER] nsd binding to 1 port?
I'm interested in specifics regarding whether AOLServer can
bind itself to
more than one port at a time.
Specifically, we would want connection threads bound to port 80, and
internal communication threads (i.e. event messages between
servers in the
cluster) bound to port 8080 (or similar).
Our internal messages among different servers in our cluster
are currently
having to wait for a free connection thread, and in times of
high traffic,
this backup causes rather critical problems.
Other solutions/suggestions are quite welcome, right now
we're trying to
figure out if this is possible in AOLServer 3.4 (upgrading is
not an option
at this point but if that would solve it, i would like to
hear that fact)
Great thanks in avance for any information you have,
Scott LaPlante
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can
leave the Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] [Tcl-Threads] Erlang's mnesia database?
I think Solid has versions of their database server that: - can be loaded directly with the application; no separate process - are thread-safe - have support for in-memory tables http://www.solidtech.com Good luck! Jim I don't intend to go back and change that particular application now, and I'm not sure how common or generally applicable this sort of experience was. But, I *KNOW* I would have very much liked to have an easy, clean, like nsv only better in-memory RDBMS available to me for that project. So off and on I think about whether there is such a tool out there, or how I might go about building one. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Relating the Unix process ID to server logs and TCL
Hello - does anyone know an easy way to relate the Unix process ID (Linux) for an AOLServer thread back to the server.log file. Can TCL get the Unix process ID for a thread? From what I can tell, server.log entries say [PPID.xxx], where: [09/Aug/2003:11:38:33][23511.6151][-conn0-] Notice: get=tid0x84845a0, getid=6151, pid=23521 23511 is the parent process id 6151 is some internal thread ID The number I really want is the last one, 23521, so that if we see a process in top using tons of CPU time, we can figure out what that thread is running. But the only way I know to get the pid is: set unixpid [lindex [read [set fd [open /proc/self/stat r]]] 0] close $fd This seems like a lot of overhead, and we'd have to do it on every request. Any ideas? Thanks in advance! Jim -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Relating the Unix process ID to server logs and TCL
[ns_info pid] does return the parent nsd process on 3.4. On a production system, when top or ps shows an nsd process chewing up 90% of a CPU, it would be useful to relate that to server.log entries. Or something. Right now it can't be related to anything. The parent nsd's process number isn't useful in this situation. Jim IMHO, threads and processes, while related, really don't belong together on a higher level. If anything, [ns_info pid] should return the process ID of the running nsd ... and hey, it does! At least it does on AOLserver 3.5.8 ... you should check yours. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Relating the Unix process ID to server logs and TCL
Cool, thanks. We're still on the 3.2 and using 7.6 TCL, so I'll have
to fiddle this a bit. It just seemed like some form of ns_thread (or
something else) should return the pid, but I couldn't find it.
Thanks again,
JIm
On 2003.08.09, Jim Wilcoxson [EMAIL PROTECTED] wrote:
This seems like a lot of overhead, and we'd have to do it on every
request.
Any ideas?
If you really need this information, perhaps its worth writing a very
thin shared object that defines some tcl proc (getpid) and returns what
the system's getpid() syscall gives you.
== start: getpidCmd.c ==
#include sys/types.h
#include unistd.h
#include tcl.h
int
GetpidObjCmd(ClientData clientData, Tcl_Interp *interp,
int objc, Tcl_Obj *CONST objv[])
{
Tcl_SetObjResult(interp, Tcl_NewIntObj(getpid()));
return TCL_OK;
}
int
Getpid_Init(Tcl_Interp *interp)
{
Tcl_CreateObjCommand(interp, sys.getpid, GetpidObjCmd, NULL, NULL);
return TCL_OK;
}
== stop: getpidCmd.c ==
$ gcc -shared -o getpid.so -I/usr/include/tcl8.4 getpidCmd.c
%
load getpid.so
ns_adp_puts [sys.getpid]
%
WARNING: This is mostly untested I just wrote into this email. On my
Linux machine it compiles and runs just fine. It should at least get
you 99% of the way to your solution.
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Pound vs. ns_write
FWIW, I hate having a bunch of separate TCL routines to do related
things. IMO, one of the nice things about TCL is that knowing you
want to do a file thing makes it reasonable to look at [file] first,
not [openfile], [rewindfile], [truncatefile], ... Separate routines
makes it harder to find out how do I do this thing?, even if I know
how to do something related (like, if I know [ns_conn peeraddr] gives
me the remote IP, it would be reasonable to guess that [ns_conn] might
also have a way to give me the client address).
Overloading ns_conn is easy in TCL:
proc rl_conn {arg {conn }} {
global __form __url
if {[info exists __url]} {
if {$arg == url} {
return $__url
} elseif {$arg == urlc} {
return [llength [split [string range $__url 1 end] /]]
} elseif {$arg == urlv} {
# NOTE: might need to trim trailing slashes here??
return [split [string range $__url 1 end] /]
}
}
if {$arg == form} {
if {[info exists __form]} {
return $__form
}
}
return [aol_ns_conn $arg]
}
# NOTE: next rename fails when file is sourced, 2nd rename fails
# on boot; this is correct
catch {rename ns_conn aol_ns_conn}
catch {rename ns_conn {}}
rename rl_conn ns_conn
Jim
+-- On Jul 19, Nathan Folkman said:
Actually I think Dossy is proposing to add a new arg to the ns_conn
command, and not changing the current behavior of ns_conn peeraddr in
order to keep backwards compatibility. I believe the proposal was to add
something like ns_conn clientaddr. Dossy, please correct me if I'm wrong.
A totally separate command, like ns_clientaddr, would be better. It's
easy to backport a standalone command - particularly when it can be
easily implemented in Tcl alone. It's more involved to backport a new
ns_conn subcommand.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] weird 0 length problems
We had a weird problem like this with ns_shares. There was a race in the shared array variable evaluation code that occurred around 1 in 20K requests on a production server. To track it down, I had to use 2 dedicated dual-cpu servers on a 100Mbit LAN, and used ab on one server to send simultaneous requests to the other. Since you have a specific URL that fails (I didn't), you should be able to get the failure rate fast enough that you can reproduce the problem quickly. In our case, sending 10K requests per second would cause the failure within a few seconds. If the simulation runs w/o exhibiting this, then your problem might be a context issue: the browser type/version, cookies that are sent with the request, etc., though that seems unlikely since the context should be exactly the same in both requests in your example below. Good luck! Jim Hello. My AOLserver is getting a lot (about 50) requests that it serves with Content-length of 0. Most of the times, people just do reload and it serves the exact page correctly. It happens very rarely (50 times for about 20 000 requests). I'm wondering how to find the bug, since I use much C code and even more Tcl code of my own. Also, I noticed that this happens only to dynamic pages - either ADP or registered using ns_register_proc. I doubt if anyone had similar problems, so I'm wondering on how to track down the problem with this bug. Here's a sample from nslog access file: XX.XX.XX.XX - - [06/Jul/2003:09:30:00 +0200] GET /popup/14 HTTP/1.0 200 0 http://www.enposi.pl/produkty/Dyktafony; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) XX.XX.XX.XX - - [06/Jul/2003:09:30:02 +0200] GET /popup/14 HTTP/1.0 200 2873 http://www.enposi.pl/produkty/Dyktafony; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) Linux 2.2.20, AOLserver 3.4.2, Tcl 8.3.4. -- WK -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] weird 0 length problems
The thing is I can't reproduce it, even when doing ab on both the same host and another one on a 100MBit LAN. I've hit about 3k hits and nothing. You are sending concurrent requests, right? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] [ aolserver-Feature Requests-746579 ] Server instance
We still have instances where AOLServer (3.2) goes deaf for some reason. The nsd processes are there, but they aren't handling any requests. So while I think this suggestion is a good idea, it probably won't take the place of watchdog monitoring, at least not for real production sites. Are you running? and Are you answering requests? are not the same question... Jim This is a very useful feature that would be very helpful for new users especially. I think it's so good, I would prefer to see it integrated such that you trigger it with a command line flag to nsd at start time. Many of you use other tools to perform watchdog activities, which is great. But others in the community who aren't as outspoken as the experts on this list might be running one-shot web sites where this feature would reduce the complexity of monitoring and restarting AOLserver should it fail. My vote is to put it in. /s. On Saturday, May 31, 2003, at 12:07 PM, Zoran Vasiljevic wrote: On Saturday 31 May 2003 18:43, you wrote: How many lines of code and how many man-hours will it take to implement in C? How long will it take to review all the code to ensure you've neither introduced any new bugs or otherwise broken already existing code? We have it running for about 3 years already. No need to develop anything since it is already there. It does not change anything related to the main server so it needs no extra testing. It is just an C-equivalent of your shell-script and it alows you to: nsd -s -t config.tcl and forget about it. It logs worker start/stop/restart in system logfile so its easy to track. Also, as a stand-alone script or program, folks running even older AOLserver 3.3.x could take advantage of it, rather than being forced to upgrade to either 4.x or 3.5.x ... or were you planning to backport this change all the way back? Ah Seeing all those responses... I think I'll just keep our own patched nsd copy and leave all as-is. It took me more time to write all those anwers, than to write the small C-routine we're talking about. If everybody is happy with shell/daemontools/whatever there is no reason for me to do anything here, obviously. Apparently, we have a very different usage pattern than most of the other users... Cheers, Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/
Re: [AOLSERVER] BIG problem with Tcl8.4.(1|2)
It seems that if people wanted to use generally-available TCL packages inside AOL, it'd be nice if cd worked and was per-thread. Otherwise, even if the TCL internals are fixed to lock around the temporary cd fiddling it does in realpath or wherever else, ordinary TCL packages still won't work in a threaded environment if they use cd. We've run into this problem ourselves, when we had to bring external TCL scripts into the web server, for coordination scheduling purposes. Jim On Thu, Mar 27, 2003 at 10:15:33AM +0100, Zoran Vasiljevic wrote: On Wednesday 26 March 2003 20:03, you wrote: 1. Are there ANY cases where a TLS (thread local storage) cd/pwd would be the WRONG thing? E.g., could we reasonally expect there to be any cases where one thread A does pwd, gets /home/my, does cd /foo, and then thread B does pwd and EXPECTS to get /foo back as the result? Any such cases in Tcl? In AOLserver? In libgcc? Anywhere in any C libraries whatsoever? I did not understand. The scenario you've used (thread A/B) is what *is* usually happening. The thread B gets /foo and this is what is expected. Yes, of course, and it can often breaks things, right? Most users of cd/pwd seem to want to neglect the fact that the pwd state is per-process not per-thread, so perhaps it would be simpler just to give them that, thus side-stepping the locking issues. Therefore, assume a TLS cd/pwd implementation, where each and every thread has it's OWN independent current working directory. Does any code break because of this? In other words, is there any code anywhere that both, a), properly treats pwd as per-process not per-thread, and b), DEPENDS on the sharing of pwd across threads for its correct behavior? But, this is not the source of the problem. The source of the problem is that Tcl internal code remembers the process-wide current working directory in an global Tcl object and neglects to properly lock access to this object for ALL operations, in addition to blindly passing the same object to other threads. And if that object were TLS, per-thread, then the error could not occur. It might cause OTHER problems, which is what my questions above were getting at, and it might plain not be feasible to implement. But a TLS pwd would solve the blindly passing a process-wide object to other threads problem above, because it wouldn't be process-wide anymore, it'd be per-thread. Clearly from what you say it's at least a bit more complicated than that though. I probably can't contribute any more usefully to the discussion until/unless I go look at the Tcl core code you're talking about... We'd need to talk to the Tcl FS implementor and get him/them abandon the object storage in favour of simple char* or put the object into the TSD. What's the TSD? Again, I'm sorry if I misunderstood your question(s). I think that you can't protect yourself from the effects of the [cd]. You will, I wasn't trying to protect myself, I was hypothesizing about possible designs/implementations for fixing the problem in Tcl. :) The fact that some system calls apparently change pwd and then change it back when they complete is really obnoxious, because that means that for proper mt-safe operation, those system calls MUST expose a mutex which any of YOUR pwd-changing code will explicitly lock and unlock as well. This is a general obnoxious feature of multi-threaded programming with multiple libraries - a given mutex sometimes must be seen and used across ALL libraries, not just within one library. I've run into that before and solved it by overriding library functions with my own mt-safe TLS versions of those functions, which let me avoid having to try to share a single mutex across multiple non-cooperatiung libraries. Thus I immediatly wondering about the possibility of using TLS to solve the cd/pwd bugs you described... I think it would be good to know what exact spots in Tcl might be potential [cd] doers. I will try to grep the code and see what Definitely, that would be good to know. -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/
Re: [AOLSERVER] ns_cache and virtual hosting ...
Are the changes such that a cache can be either per-virtual-server or per-server? It might be useful to retain the ability to have a server-wide cache. I sat down Friday afternoon with Jim's excellent short document describing how to make a module virtual server aware, and after an hour or two ended up with a modified ns_cache that fits the bill. The necessary changes were simple. I've tested it using multiple instances of OpenACS and it seems to work fine. I. To remove yourself from this list: Send a message to [EMAIL PROTECTED] with the following text in the BODY of your message: signoff aolserver II. For a complete list of listserv options please visit: http://listserv.aol.com/ III. For more AOLserver information please visit: http://www.aolserver.com/
Re: [AOLSERVER] Nesting ns_db select
You need to use two different DB handles, since the state of 2 selects cannot
be maintained in 1 DB handle.
Can I not nest ns_db select statements?
set classes [ns_db select $dbh SELECT * FROM classes]
while {[ns_db getrow $dbh $classes]} {
ns_puts trtdClass: .../td/tr
set students [ns_db select $dbh \
SELECT * FROM students WHERE cid='[ns_set value $classes id]']
while {[ns_db getrow $dbh $students]{ {
ns_puts trtdStudent.../td/tr
}
}
or is something wrong with my code that I can't figure out yet? (the above
code was just quickly typed in for an example).
Jeremy
I. To remove yourself from this list:
Send a message to [EMAIL PROTECTED] with the following text in
the BODY of your message:
signoff aolserver
II. For a complete list of listserv options please visit:
http://listserv.aol.com/
III. For more AOLserver information please visit:
http://www.aolserver.com/
I. To remove yourself from this list:
Send a message to [EMAIL PROTECTED] with the following text in
the BODY of your message:
signoff aolserver
II. For a complete list of listserv options please visit:
http://listserv.aol.com/
III. For more AOLserver information please visit:
http://www.aolserver.com/
Re: [AOLSERVER] ns_mutex is likely causing our AOL web server to hung
Put catches around your locked code and you may find a bug, for example,
incrementing an array var that doesn't exist or unsetting an array var
that doesn't exist. Without ns_mutex calls, the code may blow up but
your server won't lock up.
Jim
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
--_=_NextPart_001_01C2C61E.F6AB3890
Content-Type: text/plain
Hey Nathan!
Here is the simplified version of the code which shows how we are using
ns_mutex in our application. Basically the proc A, is being called a lot (
more than 100 times in a minute) across the applications, and proc B is
scheduled to run every ~5 minutes. Here the primary reason for using
ns_mutex is to protect counters' values while it's being manipulated (
incremented/written/cleared) from being accessed by other threads.
Please feel free to criticize this code as much as you can!
Again we are seeing that AOLserver 3.3.1 gets into trouble after calling
this procs heavily (eventually the server goes down). By only taking out
the ns_mutex lines, we'll have no problem!. Previously we never had any
problem running these on Version 2.3.3.
In the meanwhile regarding the ns_share, what is the major issue with it
that people encourage not to use it ?
Thanks!
--Seena
#
ns_share counter_A
ns_share counter_B
ns_share -init { set counter_mutex [ns_mutex create] } counter_mutex
proc X {i} {
ns_share counter_A
ns_share counter_B
ns_share counter_mutex
ns_mutex lock $counter_mutex
incr counter_A($i) 1
incr counter_B($i) 1
ns_mutex unlock $counter_mutex
}
proc_doc Y {} {
ns_share counter_A
ns_share counter_B
ns_share counter_mutex
ns_mutex lock $counter_mutex
foreach i_index [array names counter_A] {
set temp_counter_A($i_index) $conter_A($i_index)
set temp_counter_B($i_index) $conter_B($i_index)
unset $conter_A($i_index)
unset $conter_B($i_index)
}
ns_mutex unlock $counter_mutex
## writing $temp_counter_A and $temp_counter_B arrays to database
}
#
-Original Message-
From: Nathan Folkman [mailto:[EMAIL PROTECTED]]
Sent: Friday, January 24, 2003 7:08 PM
To: [EMAIL PROTECTED]
Subject: Re: [AOLSERVER] ns_mutex lock / unlock is likely causing our AOL
webserver to...
In a message dated 1/24/2003 4:47:20 PM Eastern Standard Time,
[EMAIL PROTECTED] writes:
Any more inputs regarding this matter will greatly be appreciated.
Any chance you could provide a few snippets of code showing where you are
locking and unlocking, and the work you are doing in between? Hard to tell
what the problem is. If I had to guess, however, it sounds like you are dead
locked. Perhaps you are locking, and throwing an un-caught error, and never
unlocking? Or maybe you are just experiencing contention around your
database which is causing other requests to back up waiting for that
resource... If you can provide some more detailed information, including
anythng odd you see in the server log that would be great! Also might want
to check the SYSLOG for any database errors which could point to the
problem.
Also, have you considered upgrading to at least AOLserver 3.4.2 or even
better 3.5.1? Would need more information to know exactly what you are
trying to do, but you might be able to use the nsv_incr command for your
counters.
The nsv data structure is similiar to ns_share variables in that you can
share variables between multiple threads/interps. The nsv implementation is
a lot cleaner, and handles all the synchronization for you. Plus, as I
mentioned before, there's a nifty nsv_incr command specifically for things
like counters. ns_share is not recommended, especially when running Tcl 8.x.
- Nathan
---
Thanks Andrew for your input.
We use Solaris as well and the AOLserver seems to work fine in any other
situations except when ns_mutex comes to play. Here is more details how we
are using it.
We use ns_mutex inside a scheduled proc, which writes a cashed array of
numbers (counters) to the database. This proc is scheduled for every 5
minutes, to lock that array - so that no other process can manipulate that
array at the moment it's being written to db - writes the numbers to db,
resets the counters, and then unlock that array using ns_mutex unlock.
Notice that this array is ns_share`ed. While everything seems to function
and be happy, after the webserver gets more traffic, then we'll start seeing
that all the process that have attempted to access that array, are waiting
in the queue. At this stage the nsd process will take most of the CPU usage
and the webserver almost doesn't respond the http requests. If we stop the
traffic eventually (sometimes after a long time) the server will come back
up to a normal operation and
Re: [AOLSERVER] AOLServer 4.0 hanging when killed...
We have shutdown problems all the time with 3.4.2. Just another data point. Jim --part1_ab.2896ff19.2b66f56c_boundary Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit In a message dated 1/27/2003 3:36:18 PM Eastern Standard Time, [EMAIL PROTECTED] writes: I am not sure how to check what event threads it's actually waiting for so don't know how to track this down. Although I am willing to admit it's our fault and not aolserver's :) sounds like you might be blocked on that condition broadcast waiting for detached event threads to exit... any chance you could attach with a debugger and dig around a little? you'll want to look at sched.c around line 845. also, what os are you running on? thanks! - nathan --part1_ab.2896ff19.2b66f56c_boundary Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: 7bit HTMLFONT FACE=arial,helveticaFONT SIZE=2 FAMILY=SANSSERIF FACE=Arial LANG=0In a message dated 1/27/2003 3:36:18 PM Eastern Standard Time, [EMAIL PROTECTED] writes:BR BR BLOCKQUOTE TYPE=CITE style=BORDER-LEFT: #ff 2px solid; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px; PADDING-LEFT: 5pxI am not sure how to check what event threads it's actually waitingBR for so don't know how to track this down.nbsp; Although I am willing toBR admit it's our fault and not aolserver's :)/BLOCKQUOTEBR BR sounds like you might be blocked on that condition broadcast waiting for detached event threads to exit... any chance you could attach with a debugger and dig around a little? you'll want to look at sched.c around line 845.BR BR also, what os are you running on? thanks!BR BR - nathan/FONT/HTML --part1_ab.2896ff19.2b66f56c_boundary--
Re: [AOLSERVER] DB Pool and closed connections
IMO, it would be nice to have a way to poll the server to see if the current connection has died. But killing threads immediately when that happens would be very bad. All kinds of things could be left in inconsistent states on complex sites. Jim On Fri, 2003-01-24 at 23:58, Andrew Piskorski wrote: As far as I've ever known, nothing special happens, the query runs to completion as normal. That's certainly what I've always seen happen with Oracle. I am not familiar with the nsdb internal, but AFAIK the ns_db driver and thus Postgres (or Oracle) doesn't even have any way of KNOWING whether or not the web browser is still connected to the web server or not. But... I suspect Steve M. is wondering about the second of these threads: http://openacs.org/forums/message-view?message_id=74443 http://openacs.org/forums/message-view?message_id=74451 Andrew, Indeed I am and it got me thinking that this 'problem' would apply to any AOLServer not just OpenACS. I'm curious as to how AOLServer handles a request. Would I be right in thinking that it only notices that a request is closed when it attempts to write a result back? If this is the case then it probably wouldn't notice until the query finished and it attempted to return the result. This would imply that closing the request would have no effect on the query which would run to completion. If AOLServer monitors the request and notices that it has gone does it do anything about it in terms of the processes that are running on behalf of the request? If not then I don't see how the situation (in the second thread above) could occur - Postgres would not be able to distinguish a dead request from a live one as its running the query on behalf of AOLServer not the browser. Steve -- Steve Manning - Linux Mandrake 9.0 - Gnome 2.0 East Goscote - Leicester - UK +44 (0)116 260 5457 Reply to [EMAIL PROTECTED] - AIM: verbomania Get my public key from: wwwkeys.pgp.net 25665CAF -- There are only 10 types of people in this world Those who understand binary and those who don't --
Re: [AOLSERVER] wierd log entries
That's a server bug. It was mentioned a long time ago on the list... Jim What about the lack of a space before the 400? is that a buglet in the nslog code? or some other issue I should be concerned with? - Original Message - From: Scott Goodwin [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, January 10, 2003 8:05 AM Subject: Re: [AOLSERVER] wierd log entries Yeah, it's a bad HTTP request. 400 means that the server determined a syntax error in the request. It looks like some automated program is performing conns to your server but not sending any HTTP info over, since the GET or other HTTP method isn't being stored in the log. Something may be connecting to your system via some other protocol, or is simply opening and closing the connection. /s. On Friday, January 10, 2003, at 08:58 AM, Patrick Spence wrote: Does anyone have an idea what would cause log entries like this? 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 I have thousands and thousands... Thanks. :) (baffled) -- Patrick Spence, MIS Mayor Pharmaceutical Labs/Regency Medical Research, Ltd. 2401 South 24th Street, Phoenix, AZ 85034 [EMAIL PROTECTED] - http://www.vitamist.com *** All contents of this email message are confidential and private. Please do not forward to anyone that this message is not intended for without permission. If you have received this email in error, please contact the sender and then delete all copies of the message that you have. Thank you. ***
Re: [AOLSERVER] gzip compression
I think it is browser-dependant whether gzip decompression is performed after all data is received. Here is a quote from the Mozilla site, at http://www.mozilla.org/projects/apache/gzip/ The current Mozilla source already sends Accept-encoding: gzip and can do a streaming decompression of HTML data received with Content-encoding: gzip. gzip compression is inherently stream-oriented, so the decompression can/should be stream-oriented as well. One effect you may be seeing is that if you have a web page with unconstrained tables, the browser will have to download the entire page, decompress it, and layout the entire table before anything can be displayed. This happens with or without gzip. But if you are on a very slow client omputer and have a fast connection, the gzip time could outweigh the uncompressed data transfer time and make the page display slower. I don't know whether MSIE decompresses on-the-fly or waits until all data is downloaded. Jim Has anyone investigated gzip transport encoding at all? I played around with the rlreturnz stuff and it's nice and it works, but my main problem with gzip encoded content is that the browser waits until it's downloaded the entire file before it decompresses the data and begins rendering the page. A side-effect is that it makes pages appear to take longer to load, even though they take less time to download. Of course it depends on the content that was compressed in the first place, but for HTML pages it's a noticeable difference, which is why I never adopted the use of the gzip content encoding. The other possibility I read about was using gzip for HTTP transfer encoding. So at the protocol level it would compress and stream the data (I believe...). I assume this would be pretty much transparent to the generation of HTML (or any other content) and stuffing it down the pipe and then the browser decompressing and rendering the page as it did before (without the noticeable delay). I have no idea what, if any, web browsers support this yet. Anyone got a clue or two they can share about this? - Gabriel
Re: [AOLSERVER] gzip compression
Before investing a lot of time/energy into storing the gzip results, someone may want to do some benchmarks. For a low-volume site, nothing matters. On a high-volume site, storing a large cache of gzip results on disk will put more pressure on the memory subsystem. If someone decides that storing the gzip results on disk is a good thing, the cache size and all that should probably be a config parameter, and an option for disabling it altogether would be good (cache size of zero maybe). Static file handling would also need to check the mime type and not compress images and other already-compressed data. Jim ... - In Fastpath (i.e., the static file code) do an internal-redirect to a compressed file on disk, creating it when it doesn't exist or is stale (i.e., stat both original and compressed each request which should be fine).
Re: [AOLSERVER] gzip compression
Happy New Year Nathan! Check here: http://www.rubylane.com/public/rlreturnz/rlreturnz.c If it has a performance impact, we have not noticed it. There are some browser issues, for example, with Netscape 3 on a framed site. My theory is that Netscape's gzip decompression routine has issues when multiple frames are being decompressed and rendered simultaneously. Most of the time a frameset page will display correctly, but occassionally it will goof and crash the browser or render a blank page. There may also be issues with certain version of MSIE for the Mac. Once in a blue moon a Mac user will tell us they cannot use our site, and they all report the same details. But we have staff that uses our site all the time with a Mac and doesn't have problems, so I believe it is only specific versions that have trouble. Disabling gzip compression for MSIE Mac users seemed to help some users, but did not make things work 100%, so it very well could be frame related. Jim Happy New Year! I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks! - Nathan --part1_da.219d8f87.2b471f53_boundary Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: 7bit HTMLFONT FACE=arial,helveticaFONT COLOR=#00 FACE=Arial FAMILY=SANSSERIF SIZE=2Happy New Year!BR BR I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks!BR BR - Nathan/FONTFONT COLOR=#00 FACE=Arial FAMILY=SANSSERIF SIZE=2/FONT/HTML --part1_da.219d8f87.2b471f53_boundary--
Re: [AOLSERVER] Config File Improvements
Here's an idea: make a directory like config for the nsd stuff. Have a main file there, plus a bunch of module config files. All of the module config files get sourced by a loop in the main file. This would eliminate having to list the modules in the main file and also have a config section for them. To delete a module, rename the config file. To add a module, copy in the config file, or touch it to create an empty config file. It would also let developers publish config files for their modules/extensions, if required. One specific area that is clumsy for us is IP aliases. It'd be nice to say in 1 line: ipalias xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx:8080 xxx.xxx.xxx.xxx Doing that with the nssock config directives is messy. Jim Has anyone worked on improving the default nsd.tcl file structure? For new users, and for users wanting to try something new, the lack of documentation and default values for the different modules makes experimentation difficult. So here are some random thoughts: Move module configuration into separate files. Handle all explaination and variable setting at the top of these files, source the module file iff it is actually used in the main nsd.tcl file. Or do the same with a single file, but enclose the module sections inside ifs conditioned on whether the module is used. My overall impression is that some solution that allowed the developer to concentrate on the top part of the file for setting well named, and maybe commented, variables would be much easier to setup and maintain. --Tom Jackson
Re: [AOLSERVER] mix -g/-O among modules?
I think that you will run into problems with optimization unless you use all the gcc options to tell it that statics are volatile. Most compilers will assume that statics within a C function can only be changed by that function, but that isn't true with multiple threads executing the code. We're still using egcs-2.91.66, and it doesn't have the flag for volatile statics, so I was hesitant to compile nsd7x with optimization. I did it once and it seemed to work, but I knew there were cases where it might not work and didn't want to find out the hard way. Jim I have some questions about gcc optimization settings for AOLserver: Is there any danger or drawback to compiling nsd8x and various loadable modules with different -O settings? -O2? What -O setting do most of you here use and recomend for gcc? Gcc allows it, but is it particularly useful to specify both -g and -O? -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com
Re: [AOLSERVER] ns_getform/form.tcl change
I posted a change yesterday to form.tcl, the purpose being to ensure that temp files got deleted if the upload is aborted. But I goofed and changed ns_atclose to unlink; not good. Here is another diff. The idea is to unlink the main temp file right after creating it, before ns_conncptofp has a chance to abort, and register the atclose unlink commands right after the individual temp files are opened. Jim *** form.tcl.aolTue Aug 1 13:36:17 2000 --- form.tclSun Nov 24 08:52:55 2002 *** *** 119,120 --- 119,121 } + ns_unlink $tmpfile ns_conncptofp $fp *** *** 123,125 close $fp - ns_unlink $tmpfile } --- 124,125 *** *** 201,202 --- 201,203 } + ns_atclose ns_unlink -nocomplain $tmpfile *** *** 207,213 - ns_atclose ns_unlink -nocomplain $tmpfile - close $tmp seek $fp $end ! ns_set put $form $name.tmpfile $tmpfile --- 208,212 close $tmp seek $fp $end ! ns_set update $form $name.tmpfile $tmpfile [jim /home/nsadmin/modules/tcl]$ sum form.tcl.aol 13540 7
[AOLSERVER] ns_getform/form.tcl change
During file uploads with mime/multipart, we've noticed that if a user aborts an upload, the temp files hang around. Here's a patch to modules/tcl/form.tcl that fixes this problem by unlinking the temp files immediately after they are created. This works on Unix, but may not be portable on other platforms. IMO, there is still an outstanding problem where ns_getform returns Error: Success, but I've never figured that one out. Probably EAGAIN happening or something... Jim $ diff -c1 form.tcl newform.tcl *** form.tclSat Nov 23 19:02:26 2002 --- newform.tcl Sat Nov 23 19:11:38 2002 *** *** 119,120 --- 119,121 } + ns_unlink $tmpfile ns_conncptofp $fp *** *** 123,125 close $fp - ns_unlink $tmpfile } --- 124,125 *** *** 201,202 --- 201,203 } + ns_unlink -nocomplain $tmpfile *** *** 206,209 } - - ns_atclose ns_unlink -nocomplain $tmpfile
Re: [AOLSERVER] problems
The cd TCL command does not work in AOLServer, because all threads share the same
working directory. When executed in multiple threads simulataneously, you have a
race condition.
We do this in _init.tcl, the first modules we load with AS:
rename cd {}
That disables the cd command altogether, which is what you want, because it doesn't
work.
Jim
Hi.
I've been experiencing a couple of problems with AOLServer. Running
apachebench with a concurrency of 2 or more would eventually cause
AOLServer to crash on this script:
%
cd /usr/local/aol32/servers/server1/pages/newspapers/cache/169/
puts [glob -nocomplain 1_*.js]
%
Strangely, when I shorten the directory that it is cd'ing to, it doesn't
crash.
Also, when running this script under load, AOLServer crashes with the
following tcl error (which appears in generic/tclObj.c):
%
cd /usr/local/aol32/servers/server1/pages/newspapers/cache/169/
puts [pwd]
%
UpdateStringProc should not be invoked for type cmdName
I'm also getting the following error in production (can't reliably
reproduce it) (though I'm not sure that the two are related, as the
first stopped when I rewrote the relevant code to avoid use of glob
(though if 'cd' is the problem...)):
alloc: invalid block: 0x40100d94: 3c 40 0
Sometimes it will run for hours without crashing; other times it will
crash twice in a 5-minute period (all while under fairly significant
load). It's almost always the same memory address, if that provides any
additional insight.
Thanks for any ideas you can provide.
seth
Re: [AOLSERVER] AOLserver stats package test server available
Very cool info! Thanks for the easy peek at them. -Jim I've made Nathan's stats package available on a temporary server so you can get a look at it.
Re: [AOLSERVER] nsv needs C API Re: [AOLSERVER] ns_set,
On Saturday 09 November 2002 19:43, you wrote: Since the TCL core already has all this stuff in it about binding TCL vars to C vars, it seems plausible to use it for ns_shares and get 7.6-like performance without needing traces. Ehm... the Tcl bindings *use* traces internally, i.e. on the C-level. Okay. I am showing my ignorance then. :) Speed? Yes, I admit. it would be slower. But how much? Given the skyrocketing CPU speeds, I'm more concerned about the implementation maintainability then some-% speed penalty. The test I ran to compare 8x with 7.6x was a 10-line loop, around 400K iterations, setting 2 ns_share arrays. Because of TCL 8x compilation, I expected it to go faster, but instead, it was around 3x slower. That doesn't mean our site will be 3x slower of course, but it didn't encourage me to jump on the 8x bandwagon - that's all. We have lots of TCL code and lots of ns_shares on our site. So aside from my philosophical dislike of nsv's, there's just too much code for us to change, both time-wise and risk-wise. Is the philosophical dislike based on the Tcl-side of the matter (i.e. need for wrapper commands) or the underlying implementation? I am spoiled by having true shared variables that act like TCL variables in all aspects, ie, can be passed w/o the caller knowing whether they are shared for example, writing a ton of code with that paradigm, and then having it taken away and given a way to share data among threads, which is not a TCL variable. To me, nsv's are a very different animal. For people just starting out, it's probably not a big deal. They can write their code to whatever shared data interface is available. We have 933 ns_share statements across 407 TCL source files, so the impact of changing all that code is very high for us. I do appreciate that you have the ability to consider and/or implement alternate shared var mechanisms. I can only talk about it. :) Jim
Re: [AOLSERVER] URL case insensitivity.
Here is a TCL hack to serve files w/o case sensitivity (not tested):
ns_register_proc GET / servit
ns_register_proc POST / servit
ns_register_proc HEAD / servit
proc servit {conn ignore} {
set url [string tolower [ns_conn url]]
if {[file exists [ns_info pageroot]$url]} {
ns_returnfile 200 [ns_guesstype $url] [ns_info pageroot]$url
} else {
ns_returnnotfound
}
}
Only works for static pages though.
Jim
+-- On Nov 8, Lamar Owen said:
This may be an obvious one, but I'm trying to move a site over from an IIS
host to an AOLserver one, and the web pages link to mixed-case filenames.
However, the filenames are all actually lower case, meaning I get a lot of
404's.
Write a module that provides a Tcl binding for Ns_SetRequestUrl. Then
use a that to write a Tcl preauth filter that smashes the URL to
lowercase.
Or write a module that does the same entirely in C.
Re: [AOLSERVER] How I'll vote for core team members (long)
Since AOL has already picked its representation, it seems to me they should not vote on the community half of the core team. Not that their votes would necessarily be wrong or bad or have any nefarious intention, but more in the interest of fairness, as has been said before. Just my opinion. This brings up another question: what constitutes a voting member? Does one company get 1 vote, or does each employee at a company get a vote? For myself, if rather short term limits are put on the members anyway, I'd be satisfied with listing certain criteria to be a team member, and letting any individual meeting those criteria participate on the core team. If there are too many nominees, pick out of a hat. The assumption going in is that they are all qualified. I don't think there will be tons of people fighting over this job because of the time required. Jim On Friday, November 8, 2002, at 04:27 PM, Scott S. Goodwin wrote: Should the AOL dev team members have a vote? Absolutely. They are involved with AOLserver as individuals, they run the largest sites that use AOLserver.
Re: [AOLSERVER] How I'll vote for core team members (long)
This brings up another question: what constitutes a voting member? Does one company get 1 vote, or does each employee at a company get a vote? Each member of the core team should have a vote, regardless of company. I meant in the community, who gets a vote?
Re: [AOLSERVER] benchmarking aolserver (odd behavior)
In general, is it possible to change the AOLServer configuration at run time by using the ns_section and ns_param commands? - Gabriel No, in general you cannot change an AS config at runtime by fiddling ns_param commands. These commands are all processed at startup. For example, when you specify the max keepalive connections you want, AS mallocs a bunch of data structures at startup. These can't be reconfigured by executing ns_section or ns_param commands. Jim
Re: [AOLSERVER] Responding to SourceForge-generated mail
For the folks who do not like the SF email on the regular AS discussion list, shutting down the regular list and making it SF-only would be exactly the wrong thing to do. Personally, I wouldn't mind getting a SF email summary once per week on this list, or even better, a web link to a summary page where I could see everything that has changed in the last week. I might look at that once a week, to see if there is anything interesting. The SF mailing list format is not very conducive to informal, personal discussions about a subject, IMO, because of all the formatting and everything. I realize it is a very *complete* format, but that doesn't mean it encourages people to participate in an easy way via email. It is much less personal, to me. (But, I have to thank Dossy for getting them to at least list the responses in an order that makes sense. I use elm, so trying to get to the bottom of these SF emails every time was a real pain. I just started deleting them.) What would be cool is if the SF email contained a response link that went to a web page showing the last comment, a box for me to put in a new comment, and automagically signed me in. Click, type, submit. That I'd probably use. Maybe I am just dumb with SF and this is possible, but when I go there, I get lost. Jim I don't think anyone was suggested shutting down the AOLserver discussion list, just creating a new list where SF bug, feature and patch notifications would go, so those who really wanted to see them can get them . Personally I don't think we want to disrupt the current AOLserver discussion list. We might lose some lurkers who aren't going to bother signing up on the new list, and that would be a mistake at this point. /s. -Original Message- From: AOLserver Discussion [mailto:AOLSERVER;LISTSERV.AOL.COM] On Behalf Of Nathan Folkman Sent: Tuesday, October 22, 2002 9:37 AM To: [EMAIL PROTECTED] Subject: Re: [AOLSERVER] Responding to SourceForge-generated mail On Tuesday, October 22, 2002, at 08:57 AM, Peter M. Jansson wrote: This is a really good idea. I notice that the SF project for AOLserver has mailing lists turned off -- can we get it turned on and set up a mailing list there? Sure. I just want to make sure there aren't too many forms of primary communication. How about a quick vote? 1. Continue to use existing AOL listserv 2. Start up SF listserv and shut down AOL listserv Please let me know. Thanks! - n
Re: [AOLSERVER] Responding to SourceForge-generated mail
Has anyone ever though about using BBS software, like UBB or vBulletin? It's more like a community. SF is a development environment. Jim On Tuesday, October 22, 2002, at 10:51 AM, Scott S. Goodwin wrote: I don't think anyone was suggested shutting down the AOLserver discussion list, just creating a new list where SF bug, feature and patch notifications would go, so those who really wanted to see them can get them . Personally I don't think we want to disrupt the current AOLserver discussion list. We might lose some lurkers who aren't going to bother signing up on the new list, and that would be a mistake at this point. /s. Don't worry, I'm not going to shut down anything. Just trying to better understand what changes can be made to help better serve the community. I'd be more then happy to set up additional lists. Reading all the posts this morning makes me think that we need some better ways for everyone to communicate. Would be pretty easy for us to set up some simple polls where everyone could vote on different options. Think this might make it easier to come to a consensus. What does everyone else think? - n
Re: [AOLSERVER] Responding to SourceForge-generated mail
Yeah, I agree. Get a flood of these is not much fun. I apologize for not posting on SF. I did go there, couldn't figure out what the hell to do (even though I know I am registered there), probably couldn't remember the password I registered with anyway, so I punted and just replied to the email. I realize this is lame on my part - sorry. Jim *cough* should be on their own list *cough* -- Patrick Spence arivenATarivenDOTcom www.RandomRamblings.com www.Ariven.com - Original Message - From: Dossy [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, October 21, 2002 7:03 AM Subject: [AOLSERVER] Responding to SourceForge-generated mail All, Please do not respond to SourceForge-generated mail to the mailing list. It is not a bidirectional interface. Your responses will not be seen by the participants of the ticket. If you'd like to respond, please go to http://sf.net/tracker/index.php?group=3152 and respond directly to the ticket via the web interface. Thanks, -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
Re: [AOLSERVER] [ aolserver-Support Requests-626122 ] AOLserver 3.4.2
Do you use ns_share arrays, and are you using TCl 7.6? If so, there is
a critical bug in the server.
The following code will bomb the server at random, because of a race
condition:
ns_share x
set x(1) 1
if {[info exists x(2)]} {
}
If you execute this 1000x per second or so, the server will immediately
crash.
If this is your problem, lemme know. I have a patch to nsd/tclVar.c
Jim
www.rubylane.com
Support Requests item #626122, was opened at 2002-10-20 18:25
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=203152aid=626122group_id=3152
Category: Configuration: Other
Group: aolserver3_4
Status: Open
Resolution: None
Priority: 5
Submitted By: Thorpe Mayes (tamayes)
Assigned to: Nobody/Anonymous (nobody)
Summary: AOLserver 3.4.2 restarting spontaneously
Initial Comment:
AOLserver 3.4.2 running on linux 7.1 with solid 3.5
database.
AOLserver is restarting several times a day, but
without any pattern that I can detect.
I recently upgraded the xinetd package.
Any direction will be appreciated.
Best regards,
Thorpe Mayes
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=203152aid=626122group_id=3152
Re: [AOLSERVER] Module arrogance
Personally, I like hard failures. If I configure a database in my nsd.tcl, it seems obvious that I want a database. If the server starts anyway without one, then instead of having an immediate, obvious error to handle, I have a subtle one. I go to my home page, and the page comes back, so I assume the server is working. But 3 hours later, someone goes to a page that needs to use the database, and it dies. It'd be fine to have a config setting that says keep going if you can, maybe even with multiple values to say how severe an error can be before causing a shutdown. In fact, I think there already is something like that, or used to be. I wouldn't turn it on for critical errors like not being able to contact a database. Just my personal preference in handling important errors. An important distinction to me is whether a module decides I can never work, or I can't do what someone wants for this request or at this time. If a module is loaded and knows that it can never work right because of circumstances that aren't going to change, then I think it's appropriate for it to bomb hard. Jim www.rubylane.com I think that modules should not issue fatal errors and terminate the server unless the module determines that the error it has encountered will absolutely prevent correct operation of any part of the server. For example, the current nscp module will terminate the server with a fatal error if it cannot listen to the specified control port, but the server would be perfectly capable of processing HTTP requests (assuming nothing else went wrong), so I'd like to suggest that modules be more conservative in deciding when the server can't continue without them. Database drivers are another example -- if the driver can't connect to the DB, there's an API to return the failure, and with good engineering, you should be able to construct a server that does something reasonable even in the absence of a db connection failure, so a fatal isn't a good idea. Anybody have any feelings one way or the other?
Re: [AOLSERVER] NTPL on Linux
What I would really like to see is a better wakeup mechanism so that all of the threads don't wake up at once when a new request comes in. I think that's how it works now in many cases, though don't know for sure. The other thing that is a problem is that threads seem to service requests in round-robin order rather than last-in, first-out. The effect of this is that if a site is hit with many requests, lots of threads are started. But at least on Linux, these will never die, no matter what the time-out is set to, because all threads service requests, even if requests come in 1 every second. Jim Hi. I've just read some text about Ulrich Drepper's new implementation of posix threads on Linux. It's not stable, but still did anyone test it? I read it can start and stop 100k threads in 2 seconds on an x86 box. It requires 2.5.36 kernel, glibc 2.3 beta and gcc 3.2, so I can't test it on any of my boxes - don't have a testing machine at the moment and I don't want to make my laptop too unstable... I'm just wondering if anyone has tried AOLserver on it, if it runs, causes problems and/or SEGV. Also I'm wondering if it will speed things up on higher loads - does AOLserver 3.x rotate threads after some requests? Wouldn't make too much sense to me. Also, how often does AOLserver begin new threads? -- WK (written at Stardate 56728.4) Data typing is an illusion. Everything is a sequence of bytes. -Todd Coram
Re: [AOLSERVER] CGI error
It's probably the same as ns_getform failed: Success, which we see on our site. I think the error code mapping is incorrect in AS, at least for Linux. Jim Can anyone explain this error message for me? [18/Sep/2002:11:23:34][1790.9221][-conn2-] Error: nscgi: wait for /usr/bin/perl failed: Success
Re: [AOLSERVER] ns_queryget problem
This problem is occurring in ns_getform, so the relevant code is:
set fp
while {$fp == } {
set tmpfile [ns_tmpnam]
set fp [ns_openexcl $tmpfile]
}
ns_conncptofp $fp
The disk is not full and this problem is very sporadic so I'm pretty
confident that the directory is accessible.
I looked at ns_openexcl, and it throws an error if the file already
exists, so I don't think that's it either.
Plus I still don't understand what:
Error writing content: Success
while executing
ns_conncptofp $fp
means... Both errors have been sporadic enough - once or twice every
day or two - that we have just ignored them until it gets worse...
Jim
Error writing content: resource temporarily unavailable
while executing
ns_conncptofp $fp
This probably means your AOLserver instance doesn't have write
permission where $fp points, or the directory doesn't exist. Verify that
the AOLserver user can write to where $fp is.
--Tom Jackson
Re: [AOLSERVER] ns_queryget problem
For what it's worth, we're not using any virtual server modules and see this problem. Jim Interesting you link this to virtual servers What is your virtual server technology? Jerry At 03:32 AM 8/24/2002, you wrote: dear all, i got this error message after i setting my aolserver into several virtual web server. Before i just ran one server on the aolserver and never found this error message. is there any body knows how to fix this problem since my web after i run several virtual server is not running smoothly some of the task is missing. Error writing content: resource temporarily unavailable while executing ns_conncptofp $fp (procedure ns_getform line 25) invoked from within ns_getform (procedure ns_queryget line 2) invoked from within ns_queryget target_user invoked from within chunk: 0 of adp: /usr/aolserver/servers/photokiosk/pages/userupload.adp FYI: i'm using AOLserver 3.3ad13 TIA oetjoen
Re: [AOLSERVER] Ns_WriteConn implementation?
I just looked at the socket code, the main part being in nsd/sock.c:
---
Ns_SockSend(SOCKET sock, void *buf, int towrite, int timeout)
{
int nwrote;
nwrote = send(sock, buf, towrite, 0);
if (nwrote == -1
ns_sockerrno == EWOULDBLOCK
Ns_SockWait(sock, NS_SOCK_WRITE, timeout) == NS_OK) {
nwrote = send(sock, buf, towrite, 0);
}
return nwrote;
---
Ns_SockWait(SOCKET sock, int what, int timeout)
{
fd_set set;
struct timeval tv;
int n;
if (timeout 0) {
return NS_TIMEOUT;
}
do {
FD_ZERO(set);
FD_SET(sock, set);
tv.tv_sec = timeout;
tv.tv_usec = 0;
switch (what) {
case NS_SOCK_READ:
n = select(sock+1, set, NULL, NULL, tv);
break;
case NS_SOCK_WRITE:
n = select(sock+1, NULL, set, NULL, tv);
...
---
I think it would be very hard to impossible to block until all the
data has been received by a client browser, because that only happens
at close time. Actually, data transmission from the server occurs
even after the server closes the socket.
The code above says:
- try to write up to towrite bytes
- if you wrote anything at all, return the number written
- if you couldn't write any because the socket buffer is full, wait
until there is at least 1 byte available in the socket buffer (impl. dep)
- write however many bytes you can and return the number written.
It seems to me this should be in a loop, but maybe no one else has
access to the socket and so it's no possible for the socket state to
change after the select and before the second write.
If your socket buffer is 4K for example, you could still write a 1MB
message. You would be stuck in select most of the time. As packets
leave, the socket buffer gets more space and more data can be sent
to it.
My interpretation anyway...
Jim
I'm using AOLserer 3.3+ad13 on Solaris 8.
In the past I've been told that ns_write will block until all the data
has been received by the client browser. So, if the client is on a
paricularly slow connection, ns_write could block for a long time.
Is that really true? Or is it just that ns_write will block if the
socket send buffer is full? And if so, how do I find out how big my
TCP/IP send buffer is?
I have an application where I'm generating poetentially a whole lot of
data, which can be relatively slow, and where the client can start
making use of the data even before all of it is received. So, it sure
would be nice to ns_write out the data in chunks as we generate it,
rather than waiting and ns_return'ing the whole thing at the end.
But, is ns_write's underlying implementation using the TCP/IP socket
send buffer intelligently, or does it force a send of a separate
packet for every single call to ns_write? Even if, e.g., the socket
send buffer is large and we call ns_write with only 1 byte of data?
What steps do you recomend taking in order to limit the performance
impact of using ns_write, while still being able to stream out results
as you generate them?
Thanks!
--
Andrew Piskorski [EMAIL PROTECTED]
http://www.piskorski.com
Re: [AOLSERVER] Seeking ns_cache configuration advice
Call me dumb, but I just went to the AOL documentation page and still can't find anything on ns_cache. There are 4 ns_cache_ routines to do miscellaneous things, but nothing to explain ns_cache itself. So I grepped the source I have (3.4), and found the C routines to do anything you could possibly want with a cache, plus a new undocumented TCL routine ns_cache_keys, but no ns_cache. So I went to www.aolserver.com and looked around, figuring maybe it was a module add-on. Eventually found the module page after 5-6 clicks, but still no ns_cache. I'm not complaining, because this isn't particularly important to me, but I can't imagine a relatively new AOLServer developer figuring any of this out. I'm lost, and I've been developing on it for 8 years. Oh well... back to my 5 lines of code I guess. Jim Yeah, vaguely. I went looking for documentation etc. one time, got kinda lost, and gave up. For an interface/feature to compete with 5 lines of TCL code, it has to be pretty damn easy to use and in my face. It was taking me longer to track down information on ns_cache than it would take to enter the 5 lines of code, so I gave up. Jim, are you aware that AOLserver 3 has C API for caches that automatically flush outdated entries and automatically evict LRU entries to limit memory use, and that there's an ns_cache module that provides a Tcl layer for the C API? +-- On Jul 30, Jim Wilcoxson said: Just roll your own cache, whereever you need it, or put it in a routine:
Re: [AOLSERVER] Seeking ns_cache configuration advice
Thanks Rob. -J +-- On Jul 30, Jim Wilcoxson said: I'm not complaining, because this isn't particularly important to me, but I can't imagine a relatively new AOLServer developer figuring any of this out. I'm lost, and I've been developing on it for 8 years. The ns_cache module was part of the ArsDigita release. I think it was also downloadable separately. I don't know if it ever made it to sourceforge.
Re: [AOLSERVER] malformed bucket chain crash in 3.4
-O1 doesn't work, so I think any optimization is out.-Jim It seems the problem causing our server crashes is that I recompiled AOLServer with -O2 on Linux. When compiled with -g, the server doesn't crash, and the test case seems to be running reliably at the same speed - actually, a little faster. You might also try different -O levels to see if that works. There was (is?) a bug in IBM's xlc that generated bad code in the Tk text widget with -O, but not -g or -O2+. Different techniques are used and thus you get different code sometimes that eliminates the buggy assembly code ... The wonders of compilers. Jeff
[AOLSERVER] malformed bucket chain crash in 3.4
I narrowed this nagging server crash bug down today to a test case:
ns_share sharevar
set sharevar(1) 1
if {[info exists sharevar(2)]} {
}
ns_return 200 text/plain hi
Putting this TCL script in pageroot with no other modules loaded and
hitting it at a rate of about 800 times per second from another server
using ns_geturl in 25 threads, both machines dual processors, will
trigger the crash in a few seconds. It may be repeatable using just
one machine; I didn't try that.
Removing the subscript from the info exists variable prevents the
crash.
I'm no TCL (7.6) internals expert... Tips on where to look to fix this
are most welcome.
TIA,
Jim
Re: [AOLSERVER] ns_respond / outputheaders crashes 3.4 servery
It looked to me like AS only truncated the outputheader set (deleted its contents) when cleaning up a connection. I think this crash is specifically related to the set swap that occurs when using the -headers option. Omit that and it works fine. Unfortunately, correcting this did not fix the dreaded malformed bucket chain crash in Tcl_DeleteHashEntry. Our site is being blasted by Inktomi and is crashing every 15 minutes for the last few days... I added some code in Tcl_DeleteHashEntry just before the panic call to log the URL args being processed when this occurs. I'd welcome some tips from a TCL guru to explain how to throw a TCL error and/or get some kind of useful trace/debug information, like what table the thing is barfing on. Jim Yeah I was running into this bug about a month back. When you ns_respond, AOLServer frees the outputheaders, but [ns_conn outputheaders] still returns a valid setId, so when you deal in the output headers after an ns_respond, you crash the server. -T On Sun, 9 Jun 2002 18:35:04 -0700, Jim Wilcoxson [EMAIL PROTECTED] wrote: For an esoteric reason which I won't go into, we were doing something that occasionally caused server crashes. It caused them in 2.3.3, and it causes them in 3.4. The problem seems to be using -headers, which eventually calls Ns_ConnReplaceHeaders (return.c) from NsTclRespondCmd (tclresp.c). If the -headers argument is omitted, it works okay. Here is a test case: set setid [ns_conn outputheaders] ns_log notice tst: setid=$setid ns_set put $setid abc def ns_log notice tst: put abc:def into set $setid; doing ns_respond set newset [ns_set copy [ns_conn outputheaders]] ns_respond -status 200 -type text/plain -string hi -headers $newset ns_log notice tst: back from ns_respond #1 set setid [ns_conn outputheaders] ns_log notice tst: setid=$setid ns_set put $setid abc def ns_log notice tst: done return Here is the server log: [09/Jun/2002:18:25:44][13118.9226][-conn4-] Notice: tst: setid=t1 [09/Jun/2002:18:25:44][13118.9226][-conn4-] Notice: tst: put abc:def into set t1; doing ns_respond [09/Jun/2002:18:25:44][13118.9226][-conn4-] Notice: tst: back from ns_respond #1 [09/Jun/2002:18:25:44][13118.9226][-conn4-] Notice: tst: setid=t1 nsthread(13131) error: Ns_Pool: invalid block: 0x8c6ecf8 [09/Jun/2002:18:25:45][14779.1024][-main-] Notice: nsmain: AOLserver/3.4 starting The exact error message varies: one time it said it couldn't allocate 635xx bytes of memory. Yeah, I could see how that might be tough on our dev server... Jim
Re: [AOLSERVER] ns_respond / outputheaders crashes 3.4 serveryy
On 2002.06.10, Jim Wilcoxson [EMAIL PROTECTED] wrote: I'd welcome some tips from a TCL guru to explain how to throw a TCL error and/or get some kind of useful trace/debug information, like what table the thing is barfing on. A Tcl error from C? Ya. I wrote a dump routine. Seems to work okay, although it doesn't exactly do the TCL stack trace I want. I'm just trying to figure out if this is related to a specific thing, like setting an ns_share, or if it's a random event.
Re: [AOLSERVER] bug in ns_respond?
This bug has been around forever. It also happens, at least in 2.3.3, when
you do an ns_return, forget the TCL return, and do a 2nd ns_return. We
redefined ns_return so that if it is attempted twice, the 2nd call is
ignored because it kept crashing our server.
Jim
I ran the following in a registered proc and crashed my AOLServer:
ns_respond -status 200 -type text/html \
-string html/html -headers [ns_conn outputheaders]
set setId [ns_conn outputheaders]
for {set i 0} {$i [ns_set size $setId]} {incr i} {
ns_log debug [ns_set key $setId $i]: [ns_set value $setId $i]
}
I got something like the following as output (it differs each time):
[08/May/2002:07:07:57][31507.7176][-conn0-] Debug: ðw@ðw: ðw@ðw
So it seems like what's happening is that ns_respond is freeing the strings
that held the outputheaders, as it thinks (rightfully so) that it no longer
needs them. However, the outputheaders ns_set still points to the old
memory location, so when you deal in that set, you meander until you
stumble upon a seg fault.
Has anyone ran into this? It doesn't sit right with me to make ns_respond
close the connection after it's done, but it seems like it's kinda doing
that... just half-assed.
Thanks,
-T
AOLServer 3.4 / Linux
Re: [AOLSERVER] malformed bucket chain in Tcl_DeleteHashEntry
I ran tests before choosing 7.6, and 8x was much slower. This test is a 10-15 line TCL loop to read through a file containing around 500K lines, do 3-4 string operations, set an ns_share array entry: 2.3.3, TCL 7.4: 536 seconds 3.4, TCL 8x:374 seconds 3.4, TCL 7.6: 222 seconds 8x was 68% slower, plus it is more picky about list structures and blew up on of our scripts, might have other compatibility issues, and the internationalization stuff, from what I can tell, is going to cause issues I didn't (and don't) want to deal with. I know it's important, but from other discussions on the list it sounds half-baked. I'd gladly move if the benefit is there. Except for this one malformed bucket crash problem, I have no complaints about 7.6 and don't see the benefits of going to a slower version of TCL that is also going to give us character set heartburn... Of course, it'll be a problem for us when AS 4 is released w/o TCL 7.6 support. I understand the reasons TCL 7.6 is being dumped - it has hacks to make ns_shares work well. But we use lots of ns_shares. So we're likely to stay behind again, like we did with 2.3.3, and in a couple of years I'll post a note explaining why we are still running that ancient 3.4 release. :) Production sites don't care about bells and whistles much. We care about stability and benefits vs. costs, including the time to understand something new, implement it, have a fallback plan, etc. It kinda bugs me that just because a software group comes out with new releases of software every 3 months, everyone is expected to jump on the bandwagon. Putting bug fixes back into released versions, to some reasonable extent, is really a big help to production sites that aren't able to switch environments every 6-months to a year. Jim Jim Wilcoxson wrote: We get this error quite frequently on one of our servers - 5-10 times per day. I read somwhere about an error in the TCL 7.6 expr handling that could cause it. I dunno... There are only a few modules we run on this server that we don't run on the others, and we have reviewed them several times w/o finding a problem. Doesn't mean there isn't one obviously, but we can't find it. did not realize tcl 7.6 was being used here. i guess there could be some aolserver+tcl hashtable prob in that case.. i'd just bite the bullet and go to tcl 8.x myself. tcl 7.6 is just plain ancient. it's more than 5 years old. your choice of course, but you probabaly won't get a lot of new help w/ 7.6 these days.. -mike Jim Janine Sisk wrote: Anyone know what that error is? It stopped one of my 3.2+ad12 servers in it's tracks tonight. Is it just one of those flukes of nature, or should I worry? you should worry (to some degree) this kind of message usually means you've got memory corruption happening of some sort. either something is smashing memory of you've got some tcl extension that's not being thread safe. might be time for you to break out purify.. -mike thanks, janine -- Janine Sisk President/CEO furfly.net, LLC Mont Vernon, NH Phone: 603-672-1122
Re: [AOLSERVER] malformed bucket chain in Tcl_DeleteHashEntry
On 4/18/02 10:49 AM, Jim Wilcoxson [EMAIL PROTECTED] wrote: This test is a 10-15 line TCL loop to read through a file containing around 500K lines, do 3-4 string operations, set an ns_share array entry Can you try the same test with an nsv instead of an ns_share? Yeah, although we have 100K+ lines of TCL code with ns_shares. It wouldn't matter much whether nsv's were 100x faster - it isn't a worthwhile risk for examining and changing that many lines of code. On top of that, nsv is a broken programming paradigm IMO: source changes are required for every variable reference, which is a total PITA. I understand it was easier to implement shares this way; that doesn't mean it is necessarily better, esp considering the history of ns_shares and their use as true TCL variables. Jim
Re: [AOLSERVER] C based modules, tcl interps and memory allocators
Agree, and I'd advise that you spend a couple of days reading the TCL book. If you know C, you will pick up TCL in about 5 seconds and be proficient in a week or two. If you value your time, it's a very wise investment. When we first started our site we worried about stupid details like whether we could dynamically generate our home page in TCL or whether it would be too slow. We opted to do whatever we wanted in TCL and worry about performance after it became an obvious problem. The cool thing is, we have only had a handful of obvious problems, and often they have been solved by writing a 30-line C function to do something low-level like a string search in a very long text. Most of our other performance issues are because we sometimes write stupid database queries that cause a sequential scan or don't get optimized the way we expect. Jim I'll second Rob's note, and I'll add that you'd be well-served to abstract your storage methods so that you can switch out how you manage the storage of your data without affecting how the application runs. Then you can build a quick version that grabs stuff directly from the database, do some performance analysis to figure out where the bottlenecks lie, and then start trying various tricks to make things to faster. Pete.
Re: [AOLSERVER] C based modules, tcl interps and memory allocators
When there is ordering involved, as Jason mentions, hashes/arrays don't work so great. Random access to an arbitrary element is fast, but there has to be another data structure to remember the order. Lots of times we combine the two: put the data in hashes/arrays with keys, then keep a list containing the keys in the order you want. This lets us store lots of data with each key without keeping all of it in a list, access it randomly very fast, and also access it sequentially in order(s). Jim To answer your question, there's no way to avoid the Tcl tax in AOLserver. But maybe that's not a bad thing. The linked lists are setting off warning alarms in my head. Linked lists are terrific if most of what you do is accessing your data items in a serialized stream, but if you are going to be accessing an item here or an item there, you'll waste a lot of time traversing the list getting to the item you want (unless your lists are very short), as compared with using something else like, say, a hash. But you don't want to implement hashes, you say? As it turns out, Tcl is not just an interpreter, it's a library, and you can create hashes from C and maintain them, all without ever messing with any actual Tcl language stuff. And the Tcl libraries are pretty good about bucket management and garbage collection when you use them to manage your hashes. Since you haven't told us much about your underlying problem, I can't really say that linked lists are inappropriate as a solution, but you should take a look at the functions in the Tcl libraries related to hashes and see if they might serve your needs better. Also, I think AOLserver has C-based cache functions which allow you to dedicate a chunk of memory to a hash-based cache, which might be even more stuff you can use. (No idea about ColdStore, sorry.) Pete. On Sunday, April 14, 2002, at 10:32 PM, Jason Saunders wrote: I'm designing a site written entirely in C-based modules - much of the work the modules do involves storing and reading small items of data (in the order of 40 bytes) in linked lists. The data will also stored in a database for backup, and for more general access. The reason to store it in linked lists is that it will be faster to store it in sorted form and then read directly from main memory, rather than read it from a database, sort it, build a linked list, manipulate the data, write to the database and then free the list.
Re: [AOLSERVER] malformed bucket chain in Tcl_DeleteHashEntry
We get this error quite frequently on one of our servers - 5-10 times per day. I read somwhere about an error in the TCL 7.6 expr handling that could cause it. I dunno... There are only a few modules we run on this server that we don't run on the others, and we have reviewed them several times w/o finding a problem. Doesn't mean there isn't one obviously, but we can't find it. Jim Janine Sisk wrote: Anyone know what that error is? It stopped one of my 3.2+ad12 servers in it's tracks tonight. Is it just one of those flukes of nature, or should I worry? you should worry (to some degree) this kind of message usually means you've got memory corruption happening of some sort. either something is smashing memory of you've got some tcl extension that's not being thread safe. might be time for you to break out purify.. -mike thanks, janine -- Janine Sisk President/CEO furfly.net, LLC Mont Vernon, NH Phone: 603-672-1122
Re: [AOLSERVER] [ aolserver-Bugs-446689 ] nsreturnredirect isn't
Without trying to start a flame war, I'll make these observations: 1. Though it may not be RFC compliant to do what it is doing, it will still work on 95%+ of the browsers in use. 2. It is only an issue if the developer has a coding error. 3. Taken together, it is extremely unlikely to happen in practice. 4. Given that it is unlikely, and that AOL's server developer resources are limited, how many resources should be used in making this change, reviewing it, testing it, etc.? I would say not many. 5. I do think it would be a good idea to include a note in the documentation that there is this shortcoming and that it's something to keep in mind when writing code. To me, this is the reason for submitting a bug report like this, and I'd be happy with a documentation note as the solution if I had reported the problem. Jim I am extremely disappointed how this bug was resolved, by rejecting it. ...
Re: [AOLSERVER] Help with Dotted IP conversion
When we evaluated 3.4 performance (TCL 7.6 and 8x) vs. 2.3.3 (TCL 7.4), one thing I compared was a 10-line loop to load an ns_share array. The loop contained maybe 8-10 string operations on a string of around 200 characters, and a single set command with an ns_share array lvalue. One execution of the procedure takes something like 3 minutes, i.e., TCL 8x does one compile and then executes for several minutes. I couldn't find the exact benchmark results just now, but it was something along the lines of 3.4 w/7.6 being 80-90% faster than 2.3.3/7.4, and 3.4 w/8x being only 25% faster. So in this case, the TCL 8x compiling and faster string operations were greatly outweighed by just 1 ns_share variable being set. This performance difference is the reason we didn't migrate to 8x. Just a data point that sites making heavy use of ns_share may want to consider. Jim Ah yes, the lifespan of Tcl 7.6 has certainly been longer than many expected, or perhaps wanted ... There are so many reasons to upgrade, and very, very few not to. At the Tcl level, script compatability was near 100% (it was Tk 4.x - 8.x that was more sensitive). The main reason is speed. Check out the data at: http://wiki.tcl.tk/1611.html That indicates a 6x overall speedup from 7.6 - 8.0. I'm now working on 8.4 to push that boundary even further (already beating 8.0 by 20% and we're not done yet). Then of course there are the new features. More advanced regexp, new string functions, transparent unicode support throughout, 'binary', 'namespace', more 'file' commands, ... Recall that the last patch release of 7.6 was in January 1997! A lot has happened since then. Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/ Tcl Support and Productivity Solutions
Re: [AOLSERVER] Help with Dotted IP conversion
Thanks for the info Jeff. My understanding is that the TCL 7.6 ns_share stuff is hacked into the TCL interpreter variable handler, while the 8x ns_share routines use variable traces. I had wondered about using the C variable facility to do ns_shares but haven't messed with it. It seems possible, although there is no way to unset a C variable but an ns_share can be unset. I'm sure many more issues than this... LOL. FYI, the loop was contained in a proc in a TCL file that is loaded during startup. So I assume it was compiled. We're using the standard allocator, not -z. Jim Jim Wilcoxson wrote: ... I couldn't find the exact benchmark results just now, but it was something along the lines of 3.4 w/7.6 being 80-90% faster than 2.3.3/7.4, and 3.4 w/8x being only 25% faster. So in this case, the TCL 8x compiling and faster string operations were greatly outweighed by just 1 ns_share variable being set. This performance difference is the reason we didn't migrate to 8x. Just a data point that sites making heavy use of ns_share may want to consider. The ns_share stuff is very sensitive to the performance of the memory allocator across threads. This has fluctuated over time, and I believe the one in the current aolserver head is the best so far. Earlier 8.x versions were not as good as the 7.6 version, and the default one in Tcl was really bad for threads. I'm currently working with Jim Davidson to incorporate the much faster threaded allocator into the core for 8.4. Another thing that might be noted is that Zoran Vasiljevic (main guy doing the Tcl Thread extension) mentioned, I believe, that he had a somewhat different shared variable implementation in the Thread extension that was faster than the AOLServer one. I'm not really sure, as it was just a newsgroup comment in passing. Also, I believe you implied that you were using a proc, but it is important to note that toplevel code will be slower if large loops are used without putting them in procs. This is by design. The feeling was that toplevel code is only ever evaled once, and thus doesn't need to be compiled. However large loops would always gain a compilation benefit, so they should be in procs. Some other pointers about this at http://wiki.tcl.tk/348.html. Perhaps we should always compile loops ... Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/ Tcl Support and Productivity Solutions
Re: [AOLSERVER] bug using ns_ftruncate
Hey Rob, I just tried your idea and it works correctly if flush is added. I figured this is probably a libc bug but thought I'd post it here in case others have trouble with it. Might be a good idea to add a flush inside ns_ftruncate as a workaround. Jim +-- On Feb 22, Jim Wilcoxson said: set fd [open xyz w] puts $fd abcdefg close $fd set fd [open xyz r+] seek $fd 4 puts -nonewline $fd x What happens if you put flush $fd right here, after the puts and before the ns_ftruncate? ns_ftruncate $fd 3 close $fd return
Re: [AOLSERVER] File uploading
We had a 250MB file with around 1.5M lines of 170 bytes each. A TCL program to read this file in a ~10-line loop with a few if tests, a handful of string commands (trim, length, compare), and setting an ns_share array took 365 seconds. The exact same thing in a 20-line C program took 40 seconds. In both cases the file was in memory, so no significant physical disk I/O was involved. Doing form.tcl in C probably makes a lot of sense, IMO. Jim I'm going to look into this a bit and see if it actually is the problem. If so it may help to make form.tcl into a C module to speed things up. OK, I uploaded a 2 meg file. It took 11.16 seconds total. 11.12 seconds of that was looking for file boundaries. I simply placed time tcl commands at strategic places in form.tcl to test this. Rusty
Re: [AOLSERVER] Tcl profiling
It would be cool if the ns_stats code tracked CPU time and the amount of data a script generated. Jim Hello. I've read a bit of the AOLserver sources again. I've been wondering if someone has used TclX's profiling code in AOLserver. I've read a bit on ns_stats but this is not what I want - TclX measures CPU time as well, which is what I want most. -- WK
Re: [AOLSERVER] porting: segfault
A while back I reported that we periodically get an error: malformed bucket chain in Tcl_DeleteHashEntry Some days this problem occurs 12x per day, other times only 1. But we have noticed that when we have network connectivity problems, our server crashes more often. May be related to this seg fault?? I dunno. Jim Ok, go tthe compiler problem addressed. Now it looks like there may be problem with the old code. solaris 8/aol3.4.2/gcc3.03/ If a user drops a connection before a response comes back, it looks like it causes a segfault. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 6 (LWP 7)] 0xff142060 in t_splay () from /usr/lib/libc.so.1 (gdb) bt #0 0xff142060 in t_splay () from /usr/lib/libc.so.1 #1 0xff141ecc in t_delete () from /usr/lib/libc.so.1 #2 0xff141ad0 in realfree () from /usr/lib/libc.so.1 #3 0xff142310 in _free_unlocked () from /usr/lib/libc.so.1 #4 0xff142260 in free () from /usr/lib/libc.so.1 #5 0x13eae0 in ns_free (ptr=0x2575d8) at memory.c:141 #6 0x59324 in Ns_SetFree (set=0x2575d8) at set.c:124 #7 0x57eec in NsConnThread (arg=0x1c1d50) at serv.c:712 #8 0x13d820 in NsThreadMain (arg=0x1cfbe0) at thread.c:228 (gdb) info frame 8 Stack frame at 0xfe701d18: pc = 0x13d820 in NsThreadMain (thread.c:228); saved pc 0xff25bad8 caller of frame at 0xfe701c80 source language c. Arglist at 0xfe701d18, args: arg=0x1cfbe0 Locals at 0xfe701d18, (gdb) info frame 7 Stack frame at 0xfe701c80: pc = 0x57eec in NsConnThread (serv.c:712); saved pc 0x13d820 called by frame at 0xfe701d18, caller of frame at 0xfe701b98 source language c. Arglist at 0xfe701c80, args: arg=0x1c1d50 Locals at 0xfe701c80, (gdb) info frame 6 Stack frame at 0xfe701b98: pc = 0x59324 in Ns_SetFree (set.c:124); saved pc 0x57eec called by frame at 0xfe701c80, caller of frame at 0xfe701b20 source language c. Arglist at 0xfe701b98, args: set=0x2575d8 Locals at 0xfe701b98, (gdb) info frame 5 Stack frame at 0xfe701b20: pc = 0x13eae0 in ns_free (memory.c:141); saved pc 0x59324 called by frame at 0xfe701b98, caller of frame at 0xfe701ab0 source language c. Arglist at 0xfe701b20, args: ptr=0x2575d8 Locals at 0xfe701b20, (gdb) info frame 4 Stack frame at 0xfe701ab0: pc = 0xff142260 in free; saved pc 0x13eae0 called by frame at 0xfe701b20, caller of frame at 0xfe701a50 Arglist at 0xfe701ab0, args: Locals at 0xfe701ab0,
Re: [AOLSERVER] malformed bucket chain in Tcl_DeleteHashEntry
Thanks for the help. I'll review the C extensions that we only use on this particular server since I think it's the only one that crashes with this error. RE: rename - we use this command when the server boots but not while it's running. Do you know if crashes occur when the rename function is running, or is the fact that we have renamed procedures enough to cause a crash later at some point? Thanks, Jim Jim Wilcoxson wrote: Our server is crashing 1-5 times per day with this error in the server log: malformed bucket chain in Tcl_DeleteHashEntry the above means the tcl is trying to delete a hash table entry that is not there anymore for some reason you've probably got a malloc arena corruption or double free of some kind happening if i had to make a off the cuff guess. it you have C extensions that would be the first place to look. but it could just be a subtle 7x threading bug.. in which case. you'd have to break out purify or electric fence i know of people having this problem in tcl7.6 when using the rename command under certain situations. Does anyone have a good strategy or starting point for debugging this? Running the server with gdb and waiting for the crash isn't really an option because I'd have to watch it the whole day, plus once it breakpoints our production server would be down. We're using AS 3.4 and TCL 7x - it's faster than 8x for us, and more compatible with 2.3.3's TCL. Any suggestions are welcome. Jim
[AOLSERVER] Access log format bug w/400 (Bad Request) errors
Saw this today in my access log: XX.XX.XX.XX - - [11/Jan/2002:18:38:07 -0800] 400 534 It's a Bad Request error, but the logging is goofed up - no space after the URL. This error was caused by telneting into AS 3.4 and not sending any data. Jim
Re: [AOLSERVER] script timeout
What controls are there to limit the time a page may take to return? No limits What controls are there to kill a page that is running? No controls. If you want a TCL script to have a limit, the script has to have tests inside itself to check the limits and abort processing. It would be really cool if there were a way to set a CPU and/or real time limit for scripts from inside the script, and invoke a proc with args or something, like a signal handler. Jim
Re: [AOLSERVER] Critical filter failed
What would be even better is to just write the code right in the first place so that no error ever happens. (I tried that. I came close.) People make mistakes. Server software should do something reasonable. The problem with just closing the connection is: - the user will get connection has been reset by peer, which could be caused by several things - there will be nothing logged in the server log files, like 500 errors that would show up in log analysis, i.e., the people running the server don't even know something bad is happening Especially with complex code that may be dependent on a number of things, all code paths don't always get tested. Sure, it's bad - but it's a fact of life IMO. Yes. I am arguing that the server should always return a 500 if it reaches the end of a connection with no other results. I don't know the technical feasibility of this but it does not seem like correct behavior to return nothing to the user. Totally agree. In fact, I think there may be other cases where AS 3.4 doesn't log Bad Request errors that are internally generated by AS. I have had users do screen shots of Bad Request, yet I can't find them in my logs. Could be stupidity on my part...dunno yet. Jim I'm confused by the O/ACS is replacing the http protocol with the filter reference. The O/ACS is just using the documented ns_register_filter command for it's intended purpose. On Sunday 06 January 2002 10:52 pm, you wrote: David Walker wrote: When the error below appears wouldn't the proper response be a 500 server error? The 3.4 way of handling this is to return nothing. Error: tclop: invalid return code from filter proc 'Critical filter sec_read_security_info failed.': must be filter_ok, filter_return, or filter_break This is an uncaught error in a filter. Since the O/ACS is replacing the http protocol with the filter, you could argue that it should return 500. What would be better would be for the application to log an error that would let the developer determine if it was a bug or some other error. For instance url hacking could also give this error, or many other database errors as well. --Tom Jackson
Re: [AOLSERVER] Critical filter failed
Maybe there needs to be a flag in ns_write and friends to indicate that something has been written to the connection. If not, send out a 500 before closing the connection. To me, this seems like a general error condition not specific to filters. I imagine (but don't know for a fact) that there are other unusual error cases that may cause AS to close a connection with nothing being sent. Jim Yes. I am arguing that the server should always return a 500 if it reaches the end of a connection with no other results. I don't know the technical feasibility of this but it does not seem like correct behavior to return nothing to the user. I think the problem is that a filter can return a complete response on its own, possibly using ns_write rather than ns_return. If it does do, and then fails (I have some filters that do some housekeeping after returning the request to the visitor), then the server might erroneously issue a 500 response after a perfectly well-formed (and possibly non-error, despite the failure of the filter) response. My opinion is that the behavior of the filters is reasonable; perhaps a better solution would be to enhance the documentation to point out that this is a common error. It might be better to prevent the possibility of this from occurring, but then I think we'd have to restrict some of the functionality of filters. You can accomplish a lot of what filters do with composition, so it may be that filters aren't the most appropriate choice of mechanism for this feature of your application. Pete.
Re: [AOLSERVER] How does a module know a GET or POST request was
Pete, you said don't generate the result unless you know someone's going to use it. That's the point of having a way to poll if the browser is still there - to see if they are going to use it. We do control # of requests, but it isn't straightforward. All of the code has to be very controlled with catches around everything to ensure that if any error occurs, the count of current requests gets decremented, no error returns while the count is incremented, etc. If someone double-clicks, it's much easier to just add if client has died, stop rather than a shared data structure to see if they are running twice, and you still have the problem of how to stop the first process when the second one starts up (because of double-clicking). If someone says show me a traffic summary for the last 2 years, by week, ordered by blah, it may not be trivial to generate that kind of report. So it does hurt to keep on doing it when no one is listening. Jim But here's the thing -- what does it waste to keep running when the client is gone? For most web applications (in my opinion), I think it doesn't hurt anything to finish generating the result even when nobody's listening anymore, and it makes programming significantly more complex. If generating the result is going to cost you something, then don't generate the result unless you know someone's going to use it. If you can't do that, and it's still going to cost you something, then you need to control how many requests you'll process at any one time. Programming it this way is simple, and will work in any web application framework. I really think trying to control the dialog with the visitor, even if only to notice that they aren't listening anymore, is the wrong paradigm for web programming. Pete. On Wed, 19 Dec 2001, Jim Wilcoxson wrote: I don't know how to do it (tried a bunch of things), but agree it would be nice to know if the browser side of the connection has died because of a user hitting escape or whatever. Some processes may take several minutes. If a user gets impatient or double-clicks, it's a waste to keep running when the client is gone. Some way to poll for this would be cool...
Re: [AOLSERVER] How does a module know a GET or POST request was
I guess I'll have to look at the AOL code sometime when I get a chance. It seems like when a browser closes a connection, a select event will occur on that fd, something will get marked, and creating an ns_conn polling command that said whether the fd is closed would not be a big deal... But maybe that's not the way it happens. I think a big point not to overlook is that I could develop what you are talking about, having used AS for many years. A new user probably couldn't. So what is just nice for me becomes necessary for them. Jim I'd have to agree with most of what you wrote, although I still think you can get around most of the problems (except double-clicking) with a stern warning -- perhaps an interstitial page -- that provides an estimate of how long the process will take; those estimates may not be trivial to come up with, but I think the cost of creating the estimates needs to be compared to the cost of waste. Regarding double-clicking, the most effective solution I've seen to this is to generate a request ID embedded in the request page, so that when you accept the request, you compare the request ID to the available results, and return the result, if available. If not, you can wait until it is available. The first process doesn't have to be stopped, and the second would just be waiting for the first to complete; this only wastes a thread, but it may be cheaper than reengineering ns_conn. I really do get that there are times when it would be nice to halt processing that no one will listen to, but I think it will be more expensive to implement this feature than to engineer the UI to minimize the number of times that happens. But that's just my opinion.
Re: [AOLSERVER] We've made a code change,
I'm confused. Browsers are supposed to handle relative URL redirection, so I don't understand why a relative redirect is being changed to absolute by this routine. If it were just kept relative (i.e., original code was removed instead of adding new code), then relative redirect requests could be left alone and would do what you want/expect. Jim I don't know the code well enough to comment on that aspect of it but on the idea of using the user-supplied host name in redirect I like the idea. I use a piece of tcl code that does the same thing and it's worked great for me. On Friday 07 December 2001 02:42 pm, you wrote: At BNA we have a problem with use of AOLserver in terms of how it handles the Location parameter value. The Location is a fixed value that is a global constant for the AOLserver process. ...
