Re: [AOLSERVER] CGI error
It's probably the same as ns_getform failed: Success, which we see on our site. I think the error code mapping is incorrect in AS, at least for Linux. Jim Can anyone explain this error message for me? [18/Sep/2002:11:23:34][1790.9221][-conn2-] Error: nscgi: wait for /usr/bin/perl failed: Success
Re: [AOLSERVER] NTPL on Linux
What I would really like to see is a better wakeup mechanism so that all of the threads don't wake up at once when a new request comes in. I think that's how it works now in many cases, though don't know for sure. The other thing that is a problem is that threads seem to service requests in round-robin order rather than last-in, first-out. The effect of this is that if a site is hit with many requests, lots of threads are started. But at least on Linux, these will never die, no matter what the time-out is set to, because all threads service requests, even if requests come in 1 every second. Jim Hi. I've just read some text about Ulrich Drepper's new implementation of posix threads on Linux. It's not stable, but still did anyone test it? I read it can start and stop 100k threads in 2 seconds on an x86 box. It requires 2.5.36 kernel, glibc 2.3 beta and gcc 3.2, so I can't test it on any of my boxes - don't have a testing machine at the moment and I don't want to make my laptop too unstable... I'm just wondering if anyone has tried AOLserver on it, if it runs, causes problems and/or SEGV. Also I'm wondering if it will speed things up on higher loads - does AOLserver 3.x rotate threads after some requests? Wouldn't make too much sense to me. Also, how often does AOLserver begin new threads? -- WK (written at Stardate 56728.4) Data typing is an illusion. Everything is a sequence of bytes. -Todd Coram
Re: [AOLSERVER] Module arrogance
Personally, I like hard failures. If I configure a database in my nsd.tcl, it seems obvious that I want a database. If the server starts anyway without one, then instead of having an immediate, obvious error to handle, I have a subtle one. I go to my home page, and the page comes back, so I assume the server is working. But 3 hours later, someone goes to a page that needs to use the database, and it dies. It'd be fine to have a config setting that says keep going if you can, maybe even with multiple values to say how severe an error can be before causing a shutdown. In fact, I think there already is something like that, or used to be. I wouldn't turn it on for critical errors like not being able to contact a database. Just my personal preference in handling important errors. An important distinction to me is whether a module decides I can never work, or I can't do what someone wants for this request or at this time. If a module is loaded and knows that it can never work right because of circumstances that aren't going to change, then I think it's appropriate for it to bomb hard. Jim www.rubylane.com I think that modules should not issue fatal errors and terminate the server unless the module determines that the error it has encountered will absolutely prevent correct operation of any part of the server. For example, the current nscp module will terminate the server with a fatal error if it cannot listen to the specified control port, but the server would be perfectly capable of processing HTTP requests (assuming nothing else went wrong), so I'd like to suggest that modules be more conservative in deciding when the server can't continue without them. Database drivers are another example -- if the driver can't connect to the DB, there's an API to return the failure, and with good engineering, you should be able to construct a server that does something reasonable even in the absence of a db connection failure, so a fatal isn't a good idea. Anybody have any feelings one way or the other?
Re: [AOLSERVER] [ aolserver-Support Requests-626122 ] AOLserver 3.4.2
Do you use ns_share arrays, and are you using TCl 7.6? If so, there is
a critical bug in the server.
The following code will bomb the server at random, because of a race
condition:
ns_share x
set x(1) 1
if {[info exists x(2)]} {
}
If you execute this 1000x per second or so, the server will immediately
crash.
If this is your problem, lemme know. I have a patch to nsd/tclVar.c
Jim
www.rubylane.com
Support Requests item #626122, was opened at 2002-10-20 18:25
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=203152aid=626122group_id=3152
Category: Configuration: Other
Group: aolserver3_4
Status: Open
Resolution: None
Priority: 5
Submitted By: Thorpe Mayes (tamayes)
Assigned to: Nobody/Anonymous (nobody)
Summary: AOLserver 3.4.2 restarting spontaneously
Initial Comment:
AOLserver 3.4.2 running on linux 7.1 with solid 3.5
database.
AOLserver is restarting several times a day, but
without any pattern that I can detect.
I recently upgraded the xinetd package.
Any direction will be appreciated.
Best regards,
Thorpe Mayes
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=203152aid=626122group_id=3152
Re: [AOLSERVER] Responding to SourceForge-generated mail
Yeah, I agree. Get a flood of these is not much fun. I apologize for not posting on SF. I did go there, couldn't figure out what the hell to do (even though I know I am registered there), probably couldn't remember the password I registered with anyway, so I punted and just replied to the email. I realize this is lame on my part - sorry. Jim *cough* should be on their own list *cough* -- Patrick Spence arivenATarivenDOTcom www.RandomRamblings.com www.Ariven.com - Original Message - From: Dossy [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, October 21, 2002 7:03 AM Subject: [AOLSERVER] Responding to SourceForge-generated mail All, Please do not respond to SourceForge-generated mail to the mailing list. It is not a bidirectional interface. Your responses will not be seen by the participants of the ticket. If you'd like to respond, please go to http://sf.net/tracker/index.php?group=3152 and respond directly to the ticket via the web interface. Thanks, -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
Re: [AOLSERVER] Responding to SourceForge-generated mail
For the folks who do not like the SF email on the regular AS discussion list, shutting down the regular list and making it SF-only would be exactly the wrong thing to do. Personally, I wouldn't mind getting a SF email summary once per week on this list, or even better, a web link to a summary page where I could see everything that has changed in the last week. I might look at that once a week, to see if there is anything interesting. The SF mailing list format is not very conducive to informal, personal discussions about a subject, IMO, because of all the formatting and everything. I realize it is a very *complete* format, but that doesn't mean it encourages people to participate in an easy way via email. It is much less personal, to me. (But, I have to thank Dossy for getting them to at least list the responses in an order that makes sense. I use elm, so trying to get to the bottom of these SF emails every time was a real pain. I just started deleting them.) What would be cool is if the SF email contained a response link that went to a web page showing the last comment, a box for me to put in a new comment, and automagically signed me in. Click, type, submit. That I'd probably use. Maybe I am just dumb with SF and this is possible, but when I go there, I get lost. Jim I don't think anyone was suggested shutting down the AOLserver discussion list, just creating a new list where SF bug, feature and patch notifications would go, so those who really wanted to see them can get them . Personally I don't think we want to disrupt the current AOLserver discussion list. We might lose some lurkers who aren't going to bother signing up on the new list, and that would be a mistake at this point. /s. -Original Message- From: AOLserver Discussion [mailto:AOLSERVER;LISTSERV.AOL.COM] On Behalf Of Nathan Folkman Sent: Tuesday, October 22, 2002 9:37 AM To: [EMAIL PROTECTED] Subject: Re: [AOLSERVER] Responding to SourceForge-generated mail On Tuesday, October 22, 2002, at 08:57 AM, Peter M. Jansson wrote: This is a really good idea. I notice that the SF project for AOLserver has mailing lists turned off -- can we get it turned on and set up a mailing list there? Sure. I just want to make sure there aren't too many forms of primary communication. How about a quick vote? 1. Continue to use existing AOL listserv 2. Start up SF listserv and shut down AOL listserv Please let me know. Thanks! - n
Re: [AOLSERVER] Responding to SourceForge-generated mail
Has anyone ever though about using BBS software, like UBB or vBulletin? It's more like a community. SF is a development environment. Jim On Tuesday, October 22, 2002, at 10:51 AM, Scott S. Goodwin wrote: I don't think anyone was suggested shutting down the AOLserver discussion list, just creating a new list where SF bug, feature and patch notifications would go, so those who really wanted to see them can get them . Personally I don't think we want to disrupt the current AOLserver discussion list. We might lose some lurkers who aren't going to bother signing up on the new list, and that would be a mistake at this point. /s. Don't worry, I'm not going to shut down anything. Just trying to better understand what changes can be made to help better serve the community. I'd be more then happy to set up additional lists. Reading all the posts this morning makes me think that we need some better ways for everyone to communicate. Would be pretty easy for us to set up some simple polls where everyone could vote on different options. Think this might make it easier to come to a consensus. What does everyone else think? - n
Re: [AOLSERVER] benchmarking aolserver (odd behavior)
In general, is it possible to change the AOLServer configuration at run time by using the ns_section and ns_param commands? - Gabriel No, in general you cannot change an AS config at runtime by fiddling ns_param commands. These commands are all processed at startup. For example, when you specify the max keepalive connections you want, AS mallocs a bunch of data structures at startup. These can't be reconfigured by executing ns_section or ns_param commands. Jim
Re: [AOLSERVER] URL case insensitivity.
Here is a TCL hack to serve files w/o case sensitivity (not tested):
ns_register_proc GET / servit
ns_register_proc POST / servit
ns_register_proc HEAD / servit
proc servit {conn ignore} {
set url [string tolower [ns_conn url]]
if {[file exists [ns_info pageroot]$url]} {
ns_returnfile 200 [ns_guesstype $url] [ns_info pageroot]$url
} else {
ns_returnnotfound
}
}
Only works for static pages though.
Jim
+-- On Nov 8, Lamar Owen said:
This may be an obvious one, but I'm trying to move a site over from an IIS
host to an AOLserver one, and the web pages link to mixed-case filenames.
However, the filenames are all actually lower case, meaning I get a lot of
404's.
Write a module that provides a Tcl binding for Ns_SetRequestUrl. Then
use a that to write a Tcl preauth filter that smashes the URL to
lowercase.
Or write a module that does the same entirely in C.
Re: [AOLSERVER] How I'll vote for core team members (long)
Since AOL has already picked its representation, it seems to me they should not vote on the community half of the core team. Not that their votes would necessarily be wrong or bad or have any nefarious intention, but more in the interest of fairness, as has been said before. Just my opinion. This brings up another question: what constitutes a voting member? Does one company get 1 vote, or does each employee at a company get a vote? For myself, if rather short term limits are put on the members anyway, I'd be satisfied with listing certain criteria to be a team member, and letting any individual meeting those criteria participate on the core team. If there are too many nominees, pick out of a hat. The assumption going in is that they are all qualified. I don't think there will be tons of people fighting over this job because of the time required. Jim On Friday, November 8, 2002, at 04:27 PM, Scott S. Goodwin wrote: Should the AOL dev team members have a vote? Absolutely. They are involved with AOLserver as individuals, they run the largest sites that use AOLserver.
Re: [AOLSERVER] How I'll vote for core team members (long)
This brings up another question: what constitutes a voting member? Does one company get 1 vote, or does each employee at a company get a vote? Each member of the core team should have a vote, regardless of company. I meant in the community, who gets a vote?
Re: [AOLSERVER] nsv needs C API Re: [AOLSERVER] ns_set,
On Saturday 09 November 2002 19:43, you wrote: Since the TCL core already has all this stuff in it about binding TCL vars to C vars, it seems plausible to use it for ns_shares and get 7.6-like performance without needing traces. Ehm... the Tcl bindings *use* traces internally, i.e. on the C-level. Okay. I am showing my ignorance then. :) Speed? Yes, I admit. it would be slower. But how much? Given the skyrocketing CPU speeds, I'm more concerned about the implementation maintainability then some-% speed penalty. The test I ran to compare 8x with 7.6x was a 10-line loop, around 400K iterations, setting 2 ns_share arrays. Because of TCL 8x compilation, I expected it to go faster, but instead, it was around 3x slower. That doesn't mean our site will be 3x slower of course, but it didn't encourage me to jump on the 8x bandwagon - that's all. We have lots of TCL code and lots of ns_shares on our site. So aside from my philosophical dislike of nsv's, there's just too much code for us to change, both time-wise and risk-wise. Is the philosophical dislike based on the Tcl-side of the matter (i.e. need for wrapper commands) or the underlying implementation? I am spoiled by having true shared variables that act like TCL variables in all aspects, ie, can be passed w/o the caller knowing whether they are shared for example, writing a ton of code with that paradigm, and then having it taken away and given a way to share data among threads, which is not a TCL variable. To me, nsv's are a very different animal. For people just starting out, it's probably not a big deal. They can write their code to whatever shared data interface is available. We have 933 ns_share statements across 407 TCL source files, so the impact of changing all that code is very high for us. I do appreciate that you have the ability to consider and/or implement alternate shared var mechanisms. I can only talk about it. :) Jim
Re: [AOLSERVER] AOLserver stats package test server available
Very cool info! Thanks for the easy peek at them. -Jim I've made Nathan's stats package available on a temporary server so you can get a look at it.
Re: [AOLSERVER] problems
The cd TCL command does not work in AOLServer, because all threads share the same
working directory. When executed in multiple threads simulataneously, you have a
race condition.
We do this in _init.tcl, the first modules we load with AS:
rename cd {}
That disables the cd command altogether, which is what you want, because it doesn't
work.
Jim
Hi.
I've been experiencing a couple of problems with AOLServer. Running
apachebench with a concurrency of 2 or more would eventually cause
AOLServer to crash on this script:
%
cd /usr/local/aol32/servers/server1/pages/newspapers/cache/169/
puts [glob -nocomplain 1_*.js]
%
Strangely, when I shorten the directory that it is cd'ing to, it doesn't
crash.
Also, when running this script under load, AOLServer crashes with the
following tcl error (which appears in generic/tclObj.c):
%
cd /usr/local/aol32/servers/server1/pages/newspapers/cache/169/
puts [pwd]
%
UpdateStringProc should not be invoked for type cmdName
I'm also getting the following error in production (can't reliably
reproduce it) (though I'm not sure that the two are related, as the
first stopped when I rewrote the relevant code to avoid use of glob
(though if 'cd' is the problem...)):
alloc: invalid block: 0x40100d94: 3c 40 0
Sometimes it will run for hours without crashing; other times it will
crash twice in a 5-minute period (all while under fairly significant
load). It's almost always the same memory address, if that provides any
additional insight.
Thanks for any ideas you can provide.
seth
[AOLSERVER] ns_getform/form.tcl change
During file uploads with mime/multipart, we've noticed that if a user aborts an upload, the temp files hang around. Here's a patch to modules/tcl/form.tcl that fixes this problem by unlinking the temp files immediately after they are created. This works on Unix, but may not be portable on other platforms. IMO, there is still an outstanding problem where ns_getform returns Error: Success, but I've never figured that one out. Probably EAGAIN happening or something... Jim $ diff -c1 form.tcl newform.tcl *** form.tclSat Nov 23 19:02:26 2002 --- newform.tcl Sat Nov 23 19:11:38 2002 *** *** 119,120 --- 119,121 } + ns_unlink $tmpfile ns_conncptofp $fp *** *** 123,125 close $fp - ns_unlink $tmpfile } --- 124,125 *** *** 201,202 --- 201,203 } + ns_unlink -nocomplain $tmpfile *** *** 206,209 } - - ns_atclose ns_unlink -nocomplain $tmpfile
Re: [AOLSERVER] mix -g/-O among modules?
I think that you will run into problems with optimization unless you use all the gcc options to tell it that statics are volatile. Most compilers will assume that statics within a C function can only be changed by that function, but that isn't true with multiple threads executing the code. We're still using egcs-2.91.66, and it doesn't have the flag for volatile statics, so I was hesitant to compile nsd7x with optimization. I did it once and it seemed to work, but I knew there were cases where it might not work and didn't want to find out the hard way. Jim I have some questions about gcc optimization settings for AOLserver: Is there any danger or drawback to compiling nsd8x and various loadable modules with different -O settings? -O2? What -O setting do most of you here use and recomend for gcc? Gcc allows it, but is it particularly useful to specify both -g and -O? -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com
Re: [AOLSERVER] ns_getform/form.tcl change
I posted a change yesterday to form.tcl, the purpose being to ensure that temp files got deleted if the upload is aborted. But I goofed and changed ns_atclose to unlink; not good. Here is another diff. The idea is to unlink the main temp file right after creating it, before ns_conncptofp has a chance to abort, and register the atclose unlink commands right after the individual temp files are opened. Jim *** form.tcl.aolTue Aug 1 13:36:17 2000 --- form.tclSun Nov 24 08:52:55 2002 *** *** 119,120 --- 119,121 } + ns_unlink $tmpfile ns_conncptofp $fp *** *** 123,125 close $fp - ns_unlink $tmpfile } --- 124,125 *** *** 201,202 --- 201,203 } + ns_atclose ns_unlink -nocomplain $tmpfile *** *** 207,213 - ns_atclose ns_unlink -nocomplain $tmpfile - close $tmp seek $fp $end ! ns_set put $form $name.tmpfile $tmpfile --- 208,212 close $tmp seek $fp $end ! ns_set update $form $name.tmpfile $tmpfile [jim /home/nsadmin/modules/tcl]$ sum form.tcl.aol 13540 7
Re: [AOLSERVER] Config File Improvements
Here's an idea: make a directory like config for the nsd stuff. Have a main file there, plus a bunch of module config files. All of the module config files get sourced by a loop in the main file. This would eliminate having to list the modules in the main file and also have a config section for them. To delete a module, rename the config file. To add a module, copy in the config file, or touch it to create an empty config file. It would also let developers publish config files for their modules/extensions, if required. One specific area that is clumsy for us is IP aliases. It'd be nice to say in 1 line: ipalias xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx:8080 xxx.xxx.xxx.xxx Doing that with the nssock config directives is messy. Jim Has anyone worked on improving the default nsd.tcl file structure? For new users, and for users wanting to try something new, the lack of documentation and default values for the different modules makes experimentation difficult. So here are some random thoughts: Move module configuration into separate files. Handle all explaination and variable setting at the top of these files, source the module file iff it is actually used in the main nsd.tcl file. Or do the same with a single file, but enclose the module sections inside ifs conditioned on whether the module is used. My overall impression is that some solution that allowed the developer to concentrate on the top part of the file for setting well named, and maybe commented, variables would be much easier to setup and maintain. --Tom Jackson
Re: [AOLSERVER] gzip compression
Happy New Year Nathan! Check here: http://www.rubylane.com/public/rlreturnz/rlreturnz.c If it has a performance impact, we have not noticed it. There are some browser issues, for example, with Netscape 3 on a framed site. My theory is that Netscape's gzip decompression routine has issues when multiple frames are being decompressed and rendered simultaneously. Most of the time a frameset page will display correctly, but occassionally it will goof and crash the browser or render a blank page. There may also be issues with certain version of MSIE for the Mac. Once in a blue moon a Mac user will tell us they cannot use our site, and they all report the same details. But we have staff that uses our site all the time with a Mac and doesn't have problems, so I believe it is only specific versions that have trouble. Disabling gzip compression for MSIE Mac users seemed to help some users, but did not make things work 100%, so it very well could be frame related. Jim Happy New Year! I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks! - Nathan --part1_da.219d8f87.2b471f53_boundary Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: 7bit HTMLFONT FACE=arial,helveticaFONT COLOR=#00 FACE=Arial FAMILY=SANSSERIF SIZE=2Happy New Year!BR BR I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks!BR BR - Nathan/FONTFONT COLOR=#00 FACE=Arial FAMILY=SANSSERIF SIZE=2/FONT/HTML --part1_da.219d8f87.2b471f53_boundary--
Re: [AOLSERVER] gzip compression
Before investing a lot of time/energy into storing the gzip results, someone may want to do some benchmarks. For a low-volume site, nothing matters. On a high-volume site, storing a large cache of gzip results on disk will put more pressure on the memory subsystem. If someone decides that storing the gzip results on disk is a good thing, the cache size and all that should probably be a config parameter, and an option for disabling it altogether would be good (cache size of zero maybe). Static file handling would also need to check the mime type and not compress images and other already-compressed data. Jim ... - In Fastpath (i.e., the static file code) do an internal-redirect to a compressed file on disk, creating it when it doesn't exist or is stale (i.e., stat both original and compressed each request which should be fine).
Re: [AOLSERVER] gzip compression
I think it is browser-dependant whether gzip decompression is performed after all data is received. Here is a quote from the Mozilla site, at http://www.mozilla.org/projects/apache/gzip/ The current Mozilla source already sends Accept-encoding: gzip and can do a streaming decompression of HTML data received with Content-encoding: gzip. gzip compression is inherently stream-oriented, so the decompression can/should be stream-oriented as well. One effect you may be seeing is that if you have a web page with unconstrained tables, the browser will have to download the entire page, decompress it, and layout the entire table before anything can be displayed. This happens with or without gzip. But if you are on a very slow client omputer and have a fast connection, the gzip time could outweigh the uncompressed data transfer time and make the page display slower. I don't know whether MSIE decompresses on-the-fly or waits until all data is downloaded. Jim Has anyone investigated gzip transport encoding at all? I played around with the rlreturnz stuff and it's nice and it works, but my main problem with gzip encoded content is that the browser waits until it's downloaded the entire file before it decompresses the data and begins rendering the page. A side-effect is that it makes pages appear to take longer to load, even though they take less time to download. Of course it depends on the content that was compressed in the first place, but for HTML pages it's a noticeable difference, which is why I never adopted the use of the gzip content encoding. The other possibility I read about was using gzip for HTTP transfer encoding. So at the protocol level it would compress and stream the data (I believe...). I assume this would be pretty much transparent to the generation of HTML (or any other content) and stuffing it down the pipe and then the browser decompressing and rendering the page as it did before (without the noticeable delay). I have no idea what, if any, web browsers support this yet. Anyone got a clue or two they can share about this? - Gabriel
Re: [AOLSERVER] wierd log entries
That's a server bug. It was mentioned a long time ago on the list... Jim What about the lack of a space before the 400? is that a buglet in the nslog code? or some other issue I should be concerned with? - Original Message - From: Scott Goodwin [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, January 10, 2003 8:05 AM Subject: Re: [AOLSERVER] wierd log entries Yeah, it's a bad HTTP request. 400 means that the server determined a syntax error in the request. It looks like some automated program is performing conns to your server but not sending any HTTP info over, since the GET or other HTTP method isn't being stored in the log. Something may be connecting to your system via some other protocol, or is simply opening and closing the connection. /s. On Friday, January 10, 2003, at 08:58 AM, Patrick Spence wrote: Does anyone have an idea what would cause log entries like this? 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 195.110.73.98 - - [10/Jan/2003:07:48:37 -0700] 400 0 I have thousands and thousands... Thanks. :) (baffled) -- Patrick Spence, MIS Mayor Pharmaceutical Labs/Regency Medical Research, Ltd. 2401 South 24th Street, Phoenix, AZ 85034 [EMAIL PROTECTED] - http://www.vitamist.com *** All contents of this email message are confidential and private. Please do not forward to anyone that this message is not intended for without permission. If you have received this email in error, please contact the sender and then delete all copies of the message that you have. Thank you. ***
Re: [AOLSERVER] DB Pool and closed connections
IMO, it would be nice to have a way to poll the server to see if the current connection has died. But killing threads immediately when that happens would be very bad. All kinds of things could be left in inconsistent states on complex sites. Jim On Fri, 2003-01-24 at 23:58, Andrew Piskorski wrote: As far as I've ever known, nothing special happens, the query runs to completion as normal. That's certainly what I've always seen happen with Oracle. I am not familiar with the nsdb internal, but AFAIK the ns_db driver and thus Postgres (or Oracle) doesn't even have any way of KNOWING whether or not the web browser is still connected to the web server or not. But... I suspect Steve M. is wondering about the second of these threads: http://openacs.org/forums/message-view?message_id=74443 http://openacs.org/forums/message-view?message_id=74451 Andrew, Indeed I am and it got me thinking that this 'problem' would apply to any AOLServer not just OpenACS. I'm curious as to how AOLServer handles a request. Would I be right in thinking that it only notices that a request is closed when it attempts to write a result back? If this is the case then it probably wouldn't notice until the query finished and it attempted to return the result. This would imply that closing the request would have no effect on the query which would run to completion. If AOLServer monitors the request and notices that it has gone does it do anything about it in terms of the processes that are running on behalf of the request? If not then I don't see how the situation (in the second thread above) could occur - Postgres would not be able to distinguish a dead request from a live one as its running the query on behalf of AOLServer not the browser. Steve -- Steve Manning - Linux Mandrake 9.0 - Gnome 2.0 East Goscote - Leicester - UK +44 (0)116 260 5457 Reply to [EMAIL PROTECTED] - AIM: verbomania Get my public key from: wwwkeys.pgp.net 25665CAF -- There are only 10 types of people in this world Those who understand binary and those who don't --
Re: [AOLSERVER] ns_mutex is likely causing our AOL web server to hung
Put catches around your locked code and you may find a bug, for example,
incrementing an array var that doesn't exist or unsetting an array var
that doesn't exist. Without ns_mutex calls, the code may blow up but
your server won't lock up.
Jim
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
--_=_NextPart_001_01C2C61E.F6AB3890
Content-Type: text/plain
Hey Nathan!
Here is the simplified version of the code which shows how we are using
ns_mutex in our application. Basically the proc A, is being called a lot (
more than 100 times in a minute) across the applications, and proc B is
scheduled to run every ~5 minutes. Here the primary reason for using
ns_mutex is to protect counters' values while it's being manipulated (
incremented/written/cleared) from being accessed by other threads.
Please feel free to criticize this code as much as you can!
Again we are seeing that AOLserver 3.3.1 gets into trouble after calling
this procs heavily (eventually the server goes down). By only taking out
the ns_mutex lines, we'll have no problem!. Previously we never had any
problem running these on Version 2.3.3.
In the meanwhile regarding the ns_share, what is the major issue with it
that people encourage not to use it ?
Thanks!
--Seena
#
ns_share counter_A
ns_share counter_B
ns_share -init { set counter_mutex [ns_mutex create] } counter_mutex
proc X {i} {
ns_share counter_A
ns_share counter_B
ns_share counter_mutex
ns_mutex lock $counter_mutex
incr counter_A($i) 1
incr counter_B($i) 1
ns_mutex unlock $counter_mutex
}
proc_doc Y {} {
ns_share counter_A
ns_share counter_B
ns_share counter_mutex
ns_mutex lock $counter_mutex
foreach i_index [array names counter_A] {
set temp_counter_A($i_index) $conter_A($i_index)
set temp_counter_B($i_index) $conter_B($i_index)
unset $conter_A($i_index)
unset $conter_B($i_index)
}
ns_mutex unlock $counter_mutex
## writing $temp_counter_A and $temp_counter_B arrays to database
}
#
-Original Message-
From: Nathan Folkman [mailto:[EMAIL PROTECTED]]
Sent: Friday, January 24, 2003 7:08 PM
To: [EMAIL PROTECTED]
Subject: Re: [AOLSERVER] ns_mutex lock / unlock is likely causing our AOL
webserver to...
In a message dated 1/24/2003 4:47:20 PM Eastern Standard Time,
[EMAIL PROTECTED] writes:
Any more inputs regarding this matter will greatly be appreciated.
Any chance you could provide a few snippets of code showing where you are
locking and unlocking, and the work you are doing in between? Hard to tell
what the problem is. If I had to guess, however, it sounds like you are dead
locked. Perhaps you are locking, and throwing an un-caught error, and never
unlocking? Or maybe you are just experiencing contention around your
database which is causing other requests to back up waiting for that
resource... If you can provide some more detailed information, including
anythng odd you see in the server log that would be great! Also might want
to check the SYSLOG for any database errors which could point to the
problem.
Also, have you considered upgrading to at least AOLserver 3.4.2 or even
better 3.5.1? Would need more information to know exactly what you are
trying to do, but you might be able to use the nsv_incr command for your
counters.
The nsv data structure is similiar to ns_share variables in that you can
share variables between multiple threads/interps. The nsv implementation is
a lot cleaner, and handles all the synchronization for you. Plus, as I
mentioned before, there's a nifty nsv_incr command specifically for things
like counters. ns_share is not recommended, especially when running Tcl 8.x.
- Nathan
---
Thanks Andrew for your input.
We use Solaris as well and the AOLserver seems to work fine in any other
situations except when ns_mutex comes to play. Here is more details how we
are using it.
We use ns_mutex inside a scheduled proc, which writes a cashed array of
numbers (counters) to the database. This proc is scheduled for every 5
minutes, to lock that array - so that no other process can manipulate that
array at the moment it's being written to db - writes the numbers to db,
resets the counters, and then unlock that array using ns_mutex unlock.
Notice that this array is ns_share`ed. While everything seems to function
and be happy, after the webserver gets more traffic, then we'll start seeing
that all the process that have attempted to access that array, are waiting
in the queue. At this stage the nsd process will take most of the CPU usage
and the webserver almost doesn't respond the http requests. If we stop the
traffic eventually (sometimes after a long time) the server will come back
up to a normal operation and
Re: [AOLSERVER] AOLServer 4.0 hanging when killed...
We have shutdown problems all the time with 3.4.2. Just another data point. Jim --part1_ab.2896ff19.2b66f56c_boundary Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit In a message dated 1/27/2003 3:36:18 PM Eastern Standard Time, [EMAIL PROTECTED] writes: I am not sure how to check what event threads it's actually waiting for so don't know how to track this down. Although I am willing to admit it's our fault and not aolserver's :) sounds like you might be blocked on that condition broadcast waiting for detached event threads to exit... any chance you could attach with a debugger and dig around a little? you'll want to look at sched.c around line 845. also, what os are you running on? thanks! - nathan --part1_ab.2896ff19.2b66f56c_boundary Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: 7bit HTMLFONT FACE=arial,helveticaFONT SIZE=2 FAMILY=SANSSERIF FACE=Arial LANG=0In a message dated 1/27/2003 3:36:18 PM Eastern Standard Time, [EMAIL PROTECTED] writes:BR BR BLOCKQUOTE TYPE=CITE style=BORDER-LEFT: #ff 2px solid; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px; PADDING-LEFT: 5pxI am not sure how to check what event threads it's actually waitingBR for so don't know how to track this down.nbsp; Although I am willing toBR admit it's our fault and not aolserver's :)/BLOCKQUOTEBR BR sounds like you might be blocked on that condition broadcast waiting for detached event threads to exit... any chance you could attach with a debugger and dig around a little? you'll want to look at sched.c around line 845.BR BR also, what os are you running on? thanks!BR BR - nathan/FONT/HTML --part1_ab.2896ff19.2b66f56c_boundary--
Re: [AOLSERVER] ns_cache and virtual hosting ...
Are the changes such that a cache can be either per-virtual-server or per-server? It might be useful to retain the ability to have a server-wide cache. I sat down Friday afternoon with Jim's excellent short document describing how to make a module virtual server aware, and after an hour or two ended up with a modified ns_cache that fits the bill. The necessary changes were simple. I've tested it using multiple instances of OpenACS and it seems to work fine. I. To remove yourself from this list: Send a message to [EMAIL PROTECTED] with the following text in the BODY of your message: signoff aolserver II. For a complete list of listserv options please visit: http://listserv.aol.com/ III. For more AOLserver information please visit: http://www.aolserver.com/
Re: [AOLSERVER] Nesting ns_db select
You need to use two different DB handles, since the state of 2 selects cannot
be maintained in 1 DB handle.
Can I not nest ns_db select statements?
set classes [ns_db select $dbh SELECT * FROM classes]
while {[ns_db getrow $dbh $classes]} {
ns_puts trtdClass: .../td/tr
set students [ns_db select $dbh \
SELECT * FROM students WHERE cid='[ns_set value $classes id]']
while {[ns_db getrow $dbh $students]{ {
ns_puts trtdStudent.../td/tr
}
}
or is something wrong with my code that I can't figure out yet? (the above
code was just quickly typed in for an example).
Jeremy
I. To remove yourself from this list:
Send a message to [EMAIL PROTECTED] with the following text in
the BODY of your message:
signoff aolserver
II. For a complete list of listserv options please visit:
http://listserv.aol.com/
III. For more AOLserver information please visit:
http://www.aolserver.com/
I. To remove yourself from this list:
Send a message to [EMAIL PROTECTED] with the following text in
the BODY of your message:
signoff aolserver
II. For a complete list of listserv options please visit:
http://listserv.aol.com/
III. For more AOLserver information please visit:
http://www.aolserver.com/
Re: [AOLSERVER] BIG problem with Tcl8.4.(1|2)
It seems that if people wanted to use generally-available TCL packages inside AOL, it'd be nice if cd worked and was per-thread. Otherwise, even if the TCL internals are fixed to lock around the temporary cd fiddling it does in realpath or wherever else, ordinary TCL packages still won't work in a threaded environment if they use cd. We've run into this problem ourselves, when we had to bring external TCL scripts into the web server, for coordination scheduling purposes. Jim On Thu, Mar 27, 2003 at 10:15:33AM +0100, Zoran Vasiljevic wrote: On Wednesday 26 March 2003 20:03, you wrote: 1. Are there ANY cases where a TLS (thread local storage) cd/pwd would be the WRONG thing? E.g., could we reasonally expect there to be any cases where one thread A does pwd, gets /home/my, does cd /foo, and then thread B does pwd and EXPECTS to get /foo back as the result? Any such cases in Tcl? In AOLserver? In libgcc? Anywhere in any C libraries whatsoever? I did not understand. The scenario you've used (thread A/B) is what *is* usually happening. The thread B gets /foo and this is what is expected. Yes, of course, and it can often breaks things, right? Most users of cd/pwd seem to want to neglect the fact that the pwd state is per-process not per-thread, so perhaps it would be simpler just to give them that, thus side-stepping the locking issues. Therefore, assume a TLS cd/pwd implementation, where each and every thread has it's OWN independent current working directory. Does any code break because of this? In other words, is there any code anywhere that both, a), properly treats pwd as per-process not per-thread, and b), DEPENDS on the sharing of pwd across threads for its correct behavior? But, this is not the source of the problem. The source of the problem is that Tcl internal code remembers the process-wide current working directory in an global Tcl object and neglects to properly lock access to this object for ALL operations, in addition to blindly passing the same object to other threads. And if that object were TLS, per-thread, then the error could not occur. It might cause OTHER problems, which is what my questions above were getting at, and it might plain not be feasible to implement. But a TLS pwd would solve the blindly passing a process-wide object to other threads problem above, because it wouldn't be process-wide anymore, it'd be per-thread. Clearly from what you say it's at least a bit more complicated than that though. I probably can't contribute any more usefully to the discussion until/unless I go look at the Tcl core code you're talking about... We'd need to talk to the Tcl FS implementor and get him/them abandon the object storage in favour of simple char* or put the object into the TSD. What's the TSD? Again, I'm sorry if I misunderstood your question(s). I think that you can't protect yourself from the effects of the [cd]. You will, I wasn't trying to protect myself, I was hypothesizing about possible designs/implementations for fixing the problem in Tcl. :) The fact that some system calls apparently change pwd and then change it back when they complete is really obnoxious, because that means that for proper mt-safe operation, those system calls MUST expose a mutex which any of YOUR pwd-changing code will explicitly lock and unlock as well. This is a general obnoxious feature of multi-threaded programming with multiple libraries - a given mutex sometimes must be seen and used across ALL libraries, not just within one library. I've run into that before and solved it by overriding library functions with my own mt-safe TLS versions of those functions, which let me avoid having to try to share a single mutex across multiple non-cooperatiung libraries. Thus I immediatly wondering about the possibility of using TLS to solve the cd/pwd bugs you described... I think it would be good to know what exact spots in Tcl might be potential [cd] doers. I will try to grep the code and see what Definitely, that would be good to know. -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/
Re: [AOLSERVER] [ aolserver-Feature Requests-746579 ] Server instance
We still have instances where AOLServer (3.2) goes deaf for some reason. The nsd processes are there, but they aren't handling any requests. So while I think this suggestion is a good idea, it probably won't take the place of watchdog monitoring, at least not for real production sites. Are you running? and Are you answering requests? are not the same question... Jim This is a very useful feature that would be very helpful for new users especially. I think it's so good, I would prefer to see it integrated such that you trigger it with a command line flag to nsd at start time. Many of you use other tools to perform watchdog activities, which is great. But others in the community who aren't as outspoken as the experts on this list might be running one-shot web sites where this feature would reduce the complexity of monitoring and restarting AOLserver should it fail. My vote is to put it in. /s. On Saturday, May 31, 2003, at 12:07 PM, Zoran Vasiljevic wrote: On Saturday 31 May 2003 18:43, you wrote: How many lines of code and how many man-hours will it take to implement in C? How long will it take to review all the code to ensure you've neither introduced any new bugs or otherwise broken already existing code? We have it running for about 3 years already. No need to develop anything since it is already there. It does not change anything related to the main server so it needs no extra testing. It is just an C-equivalent of your shell-script and it alows you to: nsd -s -t config.tcl and forget about it. It logs worker start/stop/restart in system logfile so its easy to track. Also, as a stand-alone script or program, folks running even older AOLserver 3.3.x could take advantage of it, rather than being forced to upgrade to either 4.x or 3.5.x ... or were you planning to backport this change all the way back? Ah Seeing all those responses... I think I'll just keep our own patched nsd copy and leave all as-is. It took me more time to write all those anwers, than to write the small C-routine we're talking about. If everybody is happy with shell/daemontools/whatever there is no reason for me to do anything here, obviously. Apparently, we have a very different usage pattern than most of the other users... Cheers, Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list: http://www.aolserver.com/listserv.html List information and options: http://listserv.aol.com/
Re: [AOLSERVER] weird 0 length problems
We had a weird problem like this with ns_shares. There was a race in the shared array variable evaluation code that occurred around 1 in 20K requests on a production server. To track it down, I had to use 2 dedicated dual-cpu servers on a 100Mbit LAN, and used ab on one server to send simultaneous requests to the other. Since you have a specific URL that fails (I didn't), you should be able to get the failure rate fast enough that you can reproduce the problem quickly. In our case, sending 10K requests per second would cause the failure within a few seconds. If the simulation runs w/o exhibiting this, then your problem might be a context issue: the browser type/version, cookies that are sent with the request, etc., though that seems unlikely since the context should be exactly the same in both requests in your example below. Good luck! Jim Hello. My AOLserver is getting a lot (about 50) requests that it serves with Content-length of 0. Most of the times, people just do reload and it serves the exact page correctly. It happens very rarely (50 times for about 20 000 requests). I'm wondering how to find the bug, since I use much C code and even more Tcl code of my own. Also, I noticed that this happens only to dynamic pages - either ADP or registered using ns_register_proc. I doubt if anyone had similar problems, so I'm wondering on how to track down the problem with this bug. Here's a sample from nslog access file: XX.XX.XX.XX - - [06/Jul/2003:09:30:00 +0200] GET /popup/14 HTTP/1.0 200 0 http://www.enposi.pl/produkty/Dyktafony; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) XX.XX.XX.XX - - [06/Jul/2003:09:30:02 +0200] GET /popup/14 HTTP/1.0 200 2873 http://www.enposi.pl/produkty/Dyktafony; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) Linux 2.2.20, AOLserver 3.4.2, Tcl 8.3.4. -- WK -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] weird 0 length problems
The thing is I can't reproduce it, even when doing ab on both the same host and another one on a 100MBit LAN. I've hit about 3k hits and nothing. You are sending concurrent requests, right? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Pound vs. ns_write
FWIW, I hate having a bunch of separate TCL routines to do related
things. IMO, one of the nice things about TCL is that knowing you
want to do a file thing makes it reasonable to look at [file] first,
not [openfile], [rewindfile], [truncatefile], ... Separate routines
makes it harder to find out how do I do this thing?, even if I know
how to do something related (like, if I know [ns_conn peeraddr] gives
me the remote IP, it would be reasonable to guess that [ns_conn] might
also have a way to give me the client address).
Overloading ns_conn is easy in TCL:
proc rl_conn {arg {conn }} {
global __form __url
if {[info exists __url]} {
if {$arg == url} {
return $__url
} elseif {$arg == urlc} {
return [llength [split [string range $__url 1 end] /]]
} elseif {$arg == urlv} {
# NOTE: might need to trim trailing slashes here??
return [split [string range $__url 1 end] /]
}
}
if {$arg == form} {
if {[info exists __form]} {
return $__form
}
}
return [aol_ns_conn $arg]
}
# NOTE: next rename fails when file is sourced, 2nd rename fails
# on boot; this is correct
catch {rename ns_conn aol_ns_conn}
catch {rename ns_conn {}}
rename rl_conn ns_conn
Jim
+-- On Jul 19, Nathan Folkman said:
Actually I think Dossy is proposing to add a new arg to the ns_conn
command, and not changing the current behavior of ns_conn peeraddr in
order to keep backwards compatibility. I believe the proposal was to add
something like ns_conn clientaddr. Dossy, please correct me if I'm wrong.
A totally separate command, like ns_clientaddr, would be better. It's
easy to backport a standalone command - particularly when it can be
easily implemented in Tcl alone. It's more involved to backport a new
ns_conn subcommand.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Relating the Unix process ID to server logs and TCL
Cool, thanks. We're still on the 3.2 and using 7.6 TCL, so I'll have
to fiddle this a bit. It just seemed like some form of ns_thread (or
something else) should return the pid, but I couldn't find it.
Thanks again,
JIm
On 2003.08.09, Jim Wilcoxson [EMAIL PROTECTED] wrote:
This seems like a lot of overhead, and we'd have to do it on every
request.
Any ideas?
If you really need this information, perhaps its worth writing a very
thin shared object that defines some tcl proc (getpid) and returns what
the system's getpid() syscall gives you.
== start: getpidCmd.c ==
#include sys/types.h
#include unistd.h
#include tcl.h
int
GetpidObjCmd(ClientData clientData, Tcl_Interp *interp,
int objc, Tcl_Obj *CONST objv[])
{
Tcl_SetObjResult(interp, Tcl_NewIntObj(getpid()));
return TCL_OK;
}
int
Getpid_Init(Tcl_Interp *interp)
{
Tcl_CreateObjCommand(interp, sys.getpid, GetpidObjCmd, NULL, NULL);
return TCL_OK;
}
== stop: getpidCmd.c ==
$ gcc -shared -o getpid.so -I/usr/include/tcl8.4 getpidCmd.c
%
load getpid.so
ns_adp_puts [sys.getpid]
%
WARNING: This is mostly untested I just wrote into this email. On my
Linux machine it compiles and runs just fine. It should at least get
you 99% of the way to your solution.
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
[AOLSERVER] Relating the Unix process ID to server logs and TCL
Hello - does anyone know an easy way to relate the Unix process ID (Linux) for an AOLServer thread back to the server.log file. Can TCL get the Unix process ID for a thread? From what I can tell, server.log entries say [PPID.xxx], where: [09/Aug/2003:11:38:33][23511.6151][-conn0-] Notice: get=tid0x84845a0, getid=6151, pid=23521 23511 is the parent process id 6151 is some internal thread ID The number I really want is the last one, 23521, so that if we see a process in top using tons of CPU time, we can figure out what that thread is running. But the only way I know to get the pid is: set unixpid [lindex [read [set fd [open /proc/self/stat r]]] 0] close $fd This seems like a lot of overhead, and we'd have to do it on every request. Any ideas? Thanks in advance! Jim -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Relating the Unix process ID to server logs and TCL
[ns_info pid] does return the parent nsd process on 3.4. On a production system, when top or ps shows an nsd process chewing up 90% of a CPU, it would be useful to relate that to server.log entries. Or something. Right now it can't be related to anything. The parent nsd's process number isn't useful in this situation. Jim IMHO, threads and processes, while related, really don't belong together on a higher level. If anything, [ns_info pid] should return the process ID of the running nsd ... and hey, it does! At least it does on AOLserver 3.5.8 ... you should check yours. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] [Tcl-Threads] Erlang's mnesia database?
I think Solid has versions of their database server that: - can be loaded directly with the application; no separate process - are thread-safe - have support for in-memory tables http://www.solidtech.com Good luck! Jim I don't intend to go back and change that particular application now, and I'm not sure how common or generally applicable this sort of experience was. But, I *KNOW* I would have very much liked to have an easy, clean, like nsv only better in-memory RDBMS available to me for that project. So off and on I think about whether there is such a tool out there, or how I might go about building one. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd binding to 1 port?
You can easily bind to more than one port/IP address, however, I
believe all requests in 3.x are dumped into a common connection queue,
which does not solve the original problem. It sounds like you want
multiple connection queues, so that one can be overloaded yet the
other is still getting serviced regularly. That doesn't seem possible
in 3.x to my knowledge, at least not without hacking.
Jim
You definitely can do this in AOLserver 4 by loading more than one nssock
module.
In AOLserver 4 if you are using the built in 'virtual server' support, you
configure one global nssock and it listens on whatver port you configure it
to.
You can then have an individual virtual server (within the same nsd process)
listen to a different port by giving it its own server specific nssock
module.
Whether this is possible with AOLserver 3.x I don't know.
Try looking in the list archives
You could try something like this to get it to listen on port 8080:
ns_section ns/server/$servername/module/nssock8080
ns_paramhostname$hostname
;# This is not the same as your hostname
ns_paramaddress $address
;# This is not the same as your host addr
ns_parambufsize [expr 16 * 1024]
;# IO buffer size
ns_paramrcvbuf 0
;# Use OS defaults
ns_paramsndbuf 0
;# Use OS defaults
ns_paramsocktimeout 30
;# Connection timeout
ns_paramsendwait30
;# Timeout for sending of data
ns_paramrecvwait30
;# Timeout for receiving data
ns_paramclosewait 2
;# Timeout for closing socket
ns_paramkeepwait30
;# Keepalive inactivity timeout
ns_parambacklog 5
;# 5 pending connections
ns_paramport8080
;# Port for HTTP (typically 80)
ns_paramlocation
;# URL for auto-redirects (trailing slash)
ns_section ns/server/${servername}/modules
ns_paramnssock8080 ${bindir}/nssock${ext}
-Original Message-
From: AOLserver Discussion
[mailto:[EMAIL PROTECTED] Behalf
Of Scott Laplante
Sent: Monday, November 03, 2003 9:36 PM
To: [EMAIL PROTECTED]
Subject: [AOLSERVER] nsd binding to 1 port?
I'm interested in specifics regarding whether AOLServer can
bind itself to
more than one port at a time.
Specifically, we would want connection threads bound to port 80, and
internal communication threads (i.e. event messages between
servers in the
cluster) bound to port 8080 (or similar).
Our internal messages among different servers in our cluster
are currently
having to wait for a free connection thread, and in times of
high traffic,
this backup causes rather critical problems.
Other solutions/suggestions are quite welcome, right now
we're trying to
figure out if this is possible in AOLServer 3.4 (upgrading is
not an option
at this point but if that would solve it, i would like to
hear that fact)
Great thanks in avance for any information you have,
Scott LaPlante
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can
leave the Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Jim's Sunday commits and AOLserver 4.1
In a message dated 11/18/2003 12:03:27 AM Eastern Standard Time, [EMAIL PROTECTED] writes: Subject: Re: Jim's Sunday commits and AOLserver 4.1 On 2003.11.17, Jim Davidson [EMAIL PROTECTED] wrote: Glad somebody noticed the new code :) The changes to kick of 4.1 were in two areas: 1. Adding an I/O event callback interface for connections before they are queued for processing (called pre-queue). The interface is a bit tricky to use and there's no documentation or example module (yet). The general idea is to enable efficient pre-fetch of remote resources prior to running the connection, keeping the connection threads from stalling on I/O. For the ordinary nssock module, there should be no impact. For the SSL module, reads are now handled by special reader threads, not the connection threads. This may require moving some state from thread to connection and/or socket local storage. For requests that include file uploads, does a single, event-driven thread read in the entire request now, including the file data, before starting a connection thread? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] HTTP_RAW_POST_DATA equivalent in AOLSERVER TCL
Try ns_writecontent. That will write the post data to a file (I think) and then you can muck with the file data using TCL. So, for example: # get interp # for a unique file name (or use ns_mktemp, or ...) set me [ns_info interp] set fd [open /tmp/post$me w+] ns_unlink /tmp/post$me ns_writecontent $fd seek $fd 0 set postdata [read $fd] close $fd Use this instead of ns_getform or ns_form. It'll probably have TCL issues if there are nulls in the post data. Good luck! Jim Hello, I would like to ask if there is an equivalent of PHP's $HTTP_RAW_POST_DATA global variable in AOLSERVER using TCL. $HTTP_RAW_POST_DATA gets an entire form post in raw format. This allows a great deal of flexibility as developers like me can parse the data from an unconventional post (e.g. multi-mime multi-part). I tried ns_getform but it doesn't seem to work because it puts form posts into an ns_set. Thanks, Hamilton -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] how to return a custom 404 page with code 404?
Instead of putting notfound.html as your parameter, you could use
notfound.tcl and create a new tcl script with this line:
ns_returnfile 404 text/html [ns_server pageroot/global/notfound.html
Save this script as /global/notfound.tcl
Good luck!
Jim
I would like my server to return a custom html page (or even better a
tcl/adp pair) when a url is not found, with an http status code of
404.
When setting this in the config file:
ns_section ns/server/${servername}/redirects
ns_param 404 /global/notfound.html ;# Not Found error page
it does return my notfound.html page, but with a 200 status code.
I know I can do 'ns_return 404 text/html' manually somewhere, but
where would be the best place to call that? In a custom notfound
tcl/adp pair? In a filter? Or is there a cleaner way to achieve what I
want?
I only tested this on OpenACS by the way, which calls
ns_returnnotfound I believe.
til
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] how to return a custom 404 page with code 404?
And don't forget the closing bracket after pageroot, like I did! ;) Instead of putting notfound.html as your parameter, you could use notfound.tcl and create a new tcl script with this line: ns_returnfile 404 text/html [ns_server pageroot/global/notfound.html Save this script as /global/notfound.tcl Good luck! Jim I would like my server to return a custom html page (or even better a tcl/adp pair) when a url is not found, with an http status code of 404. ... -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] how to return a custom 404 page with code 404?
Because this is implemented using a general internal redirect mechanism in the server. For AS to return the 404 automagically, it would need a default http status field in the connection structure to keep track of this state, and then there would be some confusion over what to do with the default status vs explicit status when someone did: ns_returnfile 200 text/html blah which is what's is happening when your static html file is returned. I do agree that it's kinda weird/bogus that a code 200 gets sent out for static files used as the not found file, but it seems more of a documentation/how to workaround issue to me rather than needing a server hack. Another thing your TCL 404 handler could do is act as a forwarding service. For example, a friendly ISP might let a customer register a forwarding web address for a while if they leave. When a 404 occurs, the ISP looks in the forwarding table and returns a 301 - Moved http status rather than the 404. Wouldn't be possible if the server was hacked to always return 404... Jim I was just wondering if there was a more obvious way to accomplish that - why doesn't aolserver return 404 status by default when having custom error pages? Is there a reason for this or is it just oversight? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Mention of AOLserver in Feb 2004 Linux Journal.
Mathopd will have higher latency as an image server, because it is single threaded and doesn't do asynchronous disk I/O. So on a high-end server, the number of IOs/sec it can generate is limited. AS does not have this problem since it is multi-threaded. However, mathopd services requests in a very fair manner: in an ab test, the difference between minimum request time and maximum request time is very small. On AS, this is not the case, and there are relatively large variances in request service time, indicating that some threads are being starved. This could be an AS design issue or an OS scheduling/resource issue - dunno. Mathopd is definitely the faster engine, but its design limits its capacity. Mathopd with asynchronous disk I/O would be awesome. Mathopd: Concurrency Level: 15 Time taken for tests: 0.976 seconds Complete requests: 1 Failed requests:0 Total transferred: 2100210 bytes HTML transferred: 450045 bytes Requests per second:10245.90 Transfer rate: 2151.85 kb/s received Connnection Times (ms) min avg max Connect:0 0 1 Processing: 0 0 1 Total: 0 0 2 AOLServer 3.4: - Concurrency Level: 15 Time taken for tests: 1.822 seconds Complete requests: 1 Failed requests:0 Total transferred: 255 bytes HTML transferred: 45 bytes Requests per second:5488.47 Transfer rate: 1399.56 kb/s received Connnection Times (ms) min avg max Connect:0 0 2 Processing: 0 1 210 Total: 0 1 212 On Thu, Jan 08, 2004 at 09:15:49PM -0500, Lamar Owen wrote: There is a good, if inconspicuous, mention of AOLserver in the Feb LJ. On page 46, in the feature on the Magnatune record label, the statement is made: Apache 2 running PHP and OpenSSL serves all the HTML pages. When Magnatune was Slashdotted, I found that Apache could not keep up with the load for images. All HTTP image requests now are off-loaded to AOLserver, which had the lowest latency to serve images at high speeds. Later: Mathopd [which they use to serve the very large streaming audio files] has more latency than AOLserver, which is why we don't use it to serve small images. Why would anyone care about the latency of serving small images? Last I heard a human being viewing images in a browser is not exactly senstive to small latencies the way a parellel MPI program might be, after all. Or are they talking about absurdly large latency differences between Mathopd and AOLserver, like several seconds or more? -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
My personal opinion is that one of the responses to this simple question was pretty shitty, and totally inaccurate. In our experience, an image-serving copy of AOLserver with 30 threads configured runs in about 25MB, regardless of the load, and serving dynamic content on 30 threads takes 100-200MB, depending on the amount of caching you're doing with ns_shares and other types of persistent in-memory data storage, and the size of your preloaded tcl modules library. Of course, if you configure your fastpath cache to be huge or make some other drastic config change in nsd.tcl, it will show up as increased process size. 30 threads is a lot; most sites won't need that, but it all depends on the size of your OS's socket buffers, average size of the content you're serving, etc. An image-serving AOLserver, with TCL disabled, tends to be very stable memory-wise. A dynamic content server tends to grow in our experience, and I've always assumed that's because of heap fragmentation problems - not necessarily leaks. Just restart your server once a day/week. Jim --- John Shafto [EMAIL PROTECTED] wrote: Doing what? This particular machine only has a 512mb of ram, and serves mostly static content. I have bigger plans for it though. AOLserver is NOT Apache. Get used to that. I'll try to keep them straight. Thanks for the tip. Am I the only one who feels somewhat uncomfortable by this response? If I've only got 50mb of static content that it's serving up, and then pushing some db stuff through the back, what on earth would possibly make the process use 2gb+? I'd be getting worried then as well... = -- live- http://www.thedenofsin.org/ to- AIM: IMFDUP _-jupiter accepts your offer-_ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
Here's a script to dump out the memory pool info:
set text
set elname blocksize nfree nget nput nrequest nlock nwait
foreach pel [ns_info pools] {
append text \nPoolname: [lindex $pel 0]\n
set i 0
foreach el [lindex $pel 1] {
append text [lindex $elname $i]: $el\n
incr i
}
}
ns_return 200 text/plain $text
return
It seems that one way to address this memory/heap problem is to force
threads to exit after a certain amount of time idle, freeing their
entire heap area. There is code in AS to do that, but I've never
gotten it to work successfully because AS tends to round-robin the
workload to all threads. So they never idle.
If threads went to the top of a work queue when they were fininshed
instead of the bottom, and threads were always removed from the top,
the idle thread exit code would stand a chance of working. Also, a
system over-configured for 30 threads would automatically adjust
itself to use only 5-6 threads, or whatever the actual load required;
5-6 fragmented heaps is better than 30.
Jim
In a message dated 1/10/04 12:03:58 AM, [EMAIL PROTECTED] writes:
A dynamic content server tends to grow in our
experience, and I've always assumed that's because of heap
fragmentation problems - not necessarily leaks.=A0 Just restart your
server once a day/week.
=20
Yes -- memory fragementation is a problem in AOLserver which does lead to=20
large processes. Actually, it's more accurate to say we accept fragmenta=
tion=20
as a side effect of Tcl's threaded memory allocator which is a per-thread,=20
power-of-2 alloctor. This allocator essentially eliminates deadly lock=20
contention problems of most normal allocators but requires high overhead and=
leads to=20
fragmentation. You can peak into the current state of the allocator with t=
he=20
ns_info pools command.
It's possible to compile Tcl without the threaded allocator by editing Tcl's=
=20
Makefile to remove the -DUSE_THREAD_ALLOC=3D1. Then you could try alternat=
e=20
malloc packages which may provide better behavior for you app., i.e.:
normal config:
ns_malloc - Tcl_Alloc - Tcl threaded allocator -- malloc/Tcl_Alloc not=20
compatible
disabled config:
ns_malloc - Tcl_Alloc - malloc -- you can pre-load some other malloc=20
shared library
HORD is often cited as a good alternatiive: =20
http://www.cs.umass.edu/~emery/hoard/. =20
-Jim
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] nssock: server ready - resuming
These messages mean that the server received a request, but there were no free connection threads to pass the request to, ie, all of your threads were busy. The naive solution is to increase MaxThreads, but this likely won't solve your problem. For example, if your machine is out of CPU, setting MaxThreads higher will only cause more processes to be waiting for CPU, and eventually you'll get this error again. Same thing if you have used all your IO capacity on the machine. If you have a special case, where your server threads are coordinating with other processes and it takes a long time, doing sleeps, or something else where the threads are waiting a lot and not consuming resources, then bumping MaxThreads might help. Jim Hi ! I'm using AOLserver/3.3.1+ad13 and it is very slow. We thought it was slow due to OACS4 that runs on it, but now I noticed the log is full with : [20/Jan/2004:01:28:58][5962.4101][-conn1-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.6151][-conn3-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.5126][-conn2-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.4101][-conn1-] Notice: nssock: server ready - resuming [20/Jan/2004:01:28:58][5962.3076][-conn0-] Notice: nssock: server ready - resuming I counted 2793 appearences in a log of 6 days. The server is listening on port 8000 and is accessed by an apache server VirtualHost doing ProxyPass. (if that has anything to do with it). The server is quite busy - running 3 instances of ACS on AOLserver, some apache websites, zope, qmail, ftp etc. BUT the other 2 sites running on AOLserver are not slow. Does anybody have a tip or a clue about these nssock msgs? Thanks, Yuval. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver 3.4.2 configuration questions
Your servername may not be set correctly in nsd.tcl, so it isn't
looking at the enabletcl directive in your /ns/server/servername
section.
Jim
It is very strange but I don't see no 'Notice: tcl:
enabling .tcl pages'...
And it also seems that my enabbletcl true statement in
my config file has no influence!!
I can change that string to everything I want..the
server just starts and doesn't complain about even
wrong things I enter there...
Still not a clue as to what is happening. My log file
contains nothing about it.
--- Mark Page [EMAIL PROTECTED] wrote:
On the running .tcl pages part (there are others who
can comment better
on your postgres questions);
Try looking at what shows up in the server log file.
This file will
either be flowing to standard-out, or to the
configured 'serverlog' path
(defaults to $homedir/log/server.log; you've
specified homedir ==
/home/rgilaard/).
You should see a 'Notice: tcl: enabling .tcl pages'
message near the
start confirming that .tcl pages have been correctly
enabled.
Also, there may be something else going awry, which
could be identified
by entries in this log, perhaps associated with
trying to service the
.tcl page request.
Dino Vliet wrote on 1/20/2004, 11:00 AM:
Dear all,
I've installed aoserver 3.4.2 on my freebsd 4.9
laptop
for developing and testing some web applications
I'm
planning to create. The installation went ok and
I can
serve static pages. However, when I try to feed
my
system some .tcl pages under pageroot I get the
whole
page as text and not the results of the tcl
statements
I wrote.
For instance, this little hello world tcl program
doesn't appear as expected in the browser.
hello.tcl
ns_return 200 text/html html
head
titleTesting/title
/head
body
pHello, world/p
/body
/html
The output of this when I put the commands in a
.tcl
file under pageroot is:
ns_return 200 text/html html
head
titleTesting/title
/head
body
pHello, world/p
/body
/html
I then tried something similar but the results
were
the same:
ns_write HTTP/1.0 200 OK
MIME-Version: 1.0
Content-Type: text/html
ns_write html
head
titleTesting/title
/head
body
pHello, world/p
/body
/html
I also have a question regarding database access
with
the postgres driver. I have installed postgres
7.3.4
and it works fine. However, I wasn't able to find
no
postgres.so file in any directory on my system. I
thought aolserver would install it somewhere and
I
could use that location to find it. What I did
find
was a /usr/local/lib/postgresql/plpgsql.so file,
and I
added it to db section, but I don't think this
will do
the trick:-)
Here is my nsd configuration file:
# $Header:
/cvsroot/aolserver/aolserver/nsd/sample-config.tcl,v
1.2 2000/12/13 21:50:27 kriston Exp $
ns_log notice config.tcl: starting to read
config
file...
#
# Set some Tcl variables that are commonly used
throughout this file.
#
set httpport 8000
set httpsport 8443
# The hostname and address should be set to
actual
values.
set hostname localhost
set address127.0.0.1
set servername server1
set serverdesc Server Name
set homedir/home/rgilaard/
set bindir
/usr/local/aolserver/bin/
set pageroot
${homedir}/servers/${servername}/pages/
set directoryfile
index.adp,index.html,index.htm
set ext [info sharedlibextension]
# nsssl: Only loads if keyfile.pem and
certfile.pem
exist.
#set sslmodule nsssl${ext} ;#
Domestic
128-bit/1024-bit SSL.
set sslmodule nsssle${ext} ;#
Exportable
40-bit/512-bit SSL.
set sslkeyfile
${homedir}/servers/${servername}/modules/nsssl/keyfile.pem
set sslcertfile
${homedir}/servers/${servername}/modules/nsssl/certfile.pem
# nscp: Uncomment the sample password and log in
with
nsadmin, password x,
# type ns_crypt newpassword salt and put
the
encrypted string below.
set nscp_port
set nscp_addr 127.0.0.1
set nscp_user
#set nscp_user nsadmin:t2GqvvaiIUbF2: ;# sample
user=nsadmin, pw=x.
#
# Global server parameters
#
ns_section ns/parameters
ns_param home$homedir
ns_param debug false
#
# Thread library (nsthread) parameters
#
ns_section ns/threads
ns_param mutexmeter true ;# measure
lock
contention
#ns_param stacksize [expr 128*1024] ;#
Per-thread
stack size.
#
# MIME
Re: [AOLSERVER] Bug #879076 (Output header issue)
Hi - For 3.4x, look in nsd/return.c and nsd/fastpath.c; fastpath is calling Ns_ConnSetLastModifiedHeader in return.c to set Last-Modified. This routine calls Ns_ConnSetHeaders, which does an ns_set put, but it could easily call Ns_ConnCondSetHeaders, which would only do the put if the header doesn't exist. Changing the put to an update would not be a good idea IMO, because then the result would depend on whether the TCL was executed before or after fastpath. Ns_ConnCondSetHeaders seems like the better approach. The same thing could be done for Content-Type (mentioned in the online bug report). Are you executing your TCL stuff in a filter/trace? Or doing an ns_returnfile? Those are the only ways I can see where you would be executing TCL to affect the headers but still ending up in the fastpath code. I've had a similar problem in the past: I needed to do an ns_returnfile, but didn't want the Last-Modified header to be sent out at all because I didn't want the result cached. Never figured out how to do that. Good luck, Jim Hello good people, A friend has helped me comment on an AOLserver bug, but I was hoping to also seek assistance here. For reference, this is the bug: http://sourceforge.net/tracker/index.php?func=detailaid=879076group_id=3152atid=103152 To answer Dossy's question, I have tried: ns_set update [ns_conn outputheaders] key value This however still results in a duplicate header (in my case Content-Type). I have confirmed this bug in both AOLserver 3.3.1 and 3.4.2. Restating what I said in the bug report: Is there any way to workaround this from within the Tcl/ADP page? If not, does anyone know which function call this may relate to within the C API? I'd be happy to submit a patch if no one else has the time to fix it. Also, does anyone know if this has been fixed in AOLserver 4? If so, would the fix be easy to back port to 3.3.1 or 3.3.4? Thank you very much, Andrew -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] SQL error reporting
In one of the 3.x releases, I think after 3.2, AS stopped reporting
the details of SQL select errors. Instead of giving the detailed
SQL error, like expected comma after blah, AS started reporting
Query was not a statement returning rows. This isn't very useful
for error trapping and reporting.
Here's a patch to go back to the old behavior. Not sure whether this
even applies to 4x.
*** dbdrv.c.~1~ Tue Jan 16 10:14:27 2001
--- dbdrv.c Tue Dec 9 10:34:35 2003
***
*** 367,368
--- 367,369
Ns_Set *setPtr = NULL;
+ int status = NS_ERROR;
***
*** 371,377
if (driverPtr-execProc != NULL) {
! if (Ns_DbExec(handle, sql) == NS_ROWS) {
! setPtr = Ns_DbBindRow(handle);
! } else {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
}
--- 372,379
if (driverPtr-execProc != NULL) {
! status = Ns_DbExec(handle, sql);
! if (status == NS_DML) {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
+ } else if (status == NS_ROWS) {
+ setPtr = Ns_DbBindRow(handle);
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] SQL error reporting
Yes, it's for catch.
A useful development hack is to change nsadmin/modules/tcl/file.tcl so
that when an error occurs, the error info is passed to an error handler
and displayed as a web page. That way the developer doesn't have to
keep a tail running of the server log when testing and paw through it.
But the detailed error info was being wiped out by the not a statement
returning rows change. This change restores the detailed error info.
Jim
Is this only for the catch info in Tcl? In the server log it is still
reported with the orriginal server message, one error before the not
returning rows one.
Bas.
Jim Wilcoxson said:
In one of the 3.x releases, I think after 3.2, AS stopped reporting
the details of SQL select errors. Instead of giving the detailed
SQL error, like expected comma after blah, AS started reporting
Query was not a statement returning rows. This isn't very useful
for error trapping and reporting.
Here's a patch to go back to the old behavior. Not sure whether this
even applies to 4x.
*** dbdrv.c.~1~ Tue Jan 16 10:14:27 2001
--- dbdrv.c Tue Dec 9 10:34:35 2003
***
*** 367,368
--- 367,369
Ns_Set *setPtr = NULL;
+ int status = NS_ERROR;
***
*** 371,377
if (driverPtr-execProc != NULL) {
! if (Ns_DbExec(handle, sql) == NS_ROWS) {
! setPtr = Ns_DbBindRow(handle);
! } else {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
}
--- 372,379
if (driverPtr-execProc != NULL) {
! status = Ns_DbExec(handle, sql);
! if (status == NS_DML) {
Ns_DbSetException(handle, NSDB,
Query was not a statement returning rows.);
+ } else if (status == NS_ROWS) {
+ setPtr = Ns_DbBindRow(handle);
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] ADP Processing happens when...?
I made this suggestion a while ago (sending the output in an output thread). One argument against it at the time is that on a multi-processor machine, you may not be able to take advantage of all the processors. However, input and output spooling is not CPU intensive, so I don't think it's all that necessary to have multiple input and output threads running. As far as memory considerations, the same amount of memory is going to be needed whether there are N threads, each spooling 1 output request, or 1 output process spooling all N output requests. An alternative is to configure your socket output buffer size so that the majority of your output responses will fit. You'll get the same basic effect as single-process output spooling, but your OS will do all of the work. Jim That got me to thinking about how AOLserver works on responses - So a single thread handles the receiving of the request - which makes perfect sense on requests around 32K or so, which is probably 99.5% of them. What about on output? In other words, a request comes in, the single IO thread reads it, then once its finished it gets handed out to a thread for processing. What happens when its done? Does that thread do all the output on its own? In my imaginary world, I would love it if the thread handed the output data back to the central I/O thread, which could return it to the client at its own pace. AOLserver's I/O thread would get huge in terms of memory for doing this, but it _should_ act as a powerful connection multiplexor - you can imagine possibly 2x maxthreads connecting, and if the individual threads finished off the connections quickly, returning data to the central thread - well, it would pretty much obviate the need for caching proxy servers in front of AOLserver, right? If it already works this way - then architectural bits like that should be showcased so that more users choose AOLserver. If it doesn't - does what I say make sense? Would it have too much memory impact? Sorry for the ramble, your previous response and that URL got me to thinking about an idea I had had before... On Tue, 11 May 2004 15:35:40 -0400, Nathan Folkman [EMAIL PROTECTED] wrote: Exactly. The idea is to keep I/O events isolated to the main driver thread, which in theory should be more efficient. The goal is to not tie up connection threads, and have them sitting idle waiting for I/O. This model is kind of the best of both worlds - single-threaded event loop that accepts connections and consumes the request, and multi-threaded connection threads to do the actual work of the request. The ideas are similar to work that's discussed here: http://www.eecs.harvard.edu/~mdw/proj/seda/ At least this is how I remember things... ;-) - Nathan Tom Jackson wrote on 5/11/2004, 3:23 PM: You have to read all the data anyway, but now the POSTed data is read in so you don't have a tcl procedure slowly reading data and controlling the connection. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. A -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_conn without passing it around
For 3.4, it's:
Ns_Conn *conn;
/* get connection structure */
conn = Ns_TclGetConn(interp);
if (conn == NULL) {
Tcl_AppendResult(interp, NULL conn??, NULL);
return TCL_ERROR;
}
Jim
Is there a way to obtain the current connection's Ns_conn if
it's not passed into a C function explicitely? Is it accessible
somewhere globally or via a function at the C level?
Thanks for any help.
--
-- Mike
Mike Schilli
[EMAIL PROTECTED]
AOL Host Infrastructure Engineering
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] ns_set questions
Sets allow entries with duplicate keys. If duplicates are returned in
the list for this new ns_set keys, that's fine, except that a naiive
programmer can't go through the list of keys returned and fetch the
corresponding values. You'd have to do a foreach loop on the keys,
increment a counter, and use the counter to lindex into the ns_set values
result:
set values [ns_set values $myset]
set i -1
foreach key [ns_set keys $myset] {
incr i ;# put it here so that continues can be used to exit early
set val [lindex $values $i]
do whatever
}
It seems to me that this is worse than the current way of using a set
index with ns_set key and ns_set value:
set size [ns_set size $myset]
for {set i 0} {$i $size} {incr i} {
set key [ns_set key $myset $i]
set val [ns_set value $myset $i]
do whatever
}
IMO, a better approach (or another one at least) would be to return a
list of key value pairs with something like ns_set list, ie:
{{key1 val1} {key2 val2} ...}
Then a loop like this can be used:
foreach setel [ns_set list $myset] {
set key [lindex $setel 0]
set val [lindex $setel 1]
do whatever
}
Another option would be ns_set aget to return alternating keys and
values in a list that could be used as input to array set - sort of,
ignoring the multiple key issue - to convert sets to arrays, and
ns_set aset could be used to convert from arrays to sets, like:
set myset [ns_set create]
ns_set aset $myset [array get myarray]
I don't know how often these would be used in practice though, because
I think experienced TCL programmers quickly avoid using ns_sets except
where they have to, like for interfacing with smallish nsd data
structures.
Jim
On Fri, Jun 11, 2004 at 02:31:18AM -0400, Jamie Rasmussen wrote:
While adding documentation to the Wiki for ns_set (see
http://panoptic.com/wiki/aolserver/ns_set) I noticed that ns_set has ns_set
Also, I think it would be useful to add some new subcommands, ns_set keys
$setId and ns_set values $setId. These would return a list of the keys or
values in the ns_set respectively. While you can get these lists from
looping with ns_set key / value / array, it is convenient to have them as
Sounds like a good idea to me. Having to manually iterate over the
ns_set to find all matching keys always did seem awfully strange to
me.
What do people think? I'd be happy to submit a patch for the changes. Are
there any other ns_set modifications you've been wanting?
About two years ago, I thought I wanted some, so I patched the stuff I
wanted into my local AOLserver code. Ah, it was the findall and
findmatch stuff:
http://sourceforge.net/tracker/?func=detailaid=530808group_id=3152atid=303152
But then I learned what horrible performance ns_set has for large
quantities of data [I just added a note to the Wiki about that now],
which was exactly the situation I was using findall and findmatch
with. So I switched to Tcl Arrays, my need for a richer ns_set API
evaporated then and there, and I never did post my cleaned up patch.
(I still have it in my CVS though.)
--
Andrew Piskorski [EMAIL PROTECTED]
http://www.piskorski.com/
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] ANN: Searchable list archive at Gmane.org
If you are interested in adding historical stuff to the archive, I have the mailing list from March 23rd, 1996 to Oct 9, 1999. Lemme know. Jim As an added bonus (!!!), I've supplied my own personal mail archive of all the messages I've received from the list since January 2000 when the list was created, and the kind folks at Gmane have started importing them all! So, very soon you will be able to search the archive all the way back to the beginning. (I think my archive may be missing a total of 50 messages over the past 4 years. Oh well, I'm sorry about that.) I hope this new archive will be useful to folks, and hoepfully will help more people once the mailing list contents can be found via Google. I'll continue to gather everyone's feedback and try to come up with other common themes, but the next big one is, of course, the documentation which is being actively worked on. Again, I plead with everyone: help out, pitch in and contribute to the documentation! Either by helping to write it, or by making specific requests for what documentation you feel would be most useful! As usual, feel free to send requests directly to me off-list and I'll take care of the rest. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ANN: Searchable list archive at Gmane.org
Hi Dossy - sorry, your email was filtered. Because I use a published email address, I had to take the sledgehammer approach to spam filtering and toss anything that comes directly from a dsl or cable connected computer. Sad, but necessary for over 500 spams per day. The files I have are in listserve dump format, like this: = Date: Sat, 23 Mar 1996 16:37:24 -0500 Reply-To: Web Development with GNNserver [EMAIL PROTECTED] Sender: Web Development with GNNserver [EMAIL PROTECTED] From: Jeff Rawlings [EMAIL PROTECTED] March 23, 1996 Dear NaviSoft Customer: NaviSoft has always offered our customers the best web products and services available. Over the past year we have continued to refine our features and enhance our customer support. In October, we announced the launch of NaviService, which integrated both NaviPress and NaviServer into a powerful web hosting service. We are now integrating our web hosting service into GNN (Global Network Navigator), America Online's flagship Internet Service Provider, and we announce four exciting new developments: = Date: Sat, 23 Mar 1996 16:37:24 -0500 ... So I think I can reformat it to mbox w/o too much trouble. I will send you one file as a test. The archive is in 41 chunks. And I'll see about adding a whitelist to our spam filter! Jim On 2004.06.18, Andrew Piskorski [EMAIL PROTECTED] wrote: On Fri, Jun 11, 2004 at 02:50:40PM -0700, Jim Wilcoxson wrote: If you are interested in adding historical stuff to the archive, I have the mailing list from March 23rd, 1996 to Oct 9, 1999. Lemme know. That would be neat, please do add those really old archives. I replied to Jim directly off-list indicating that I was interested, but I need to know what format his archive of the mailing list is in. The Gmane.org folks need it in Unix mbox format. Jim, what format do you have the messages in? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TclX keyed list changes in HEAD
Hi - here are my thoughts on compatibility: Breaking compatibility at the TCL level is worse than breaking it at the C level. If someone wants to write a new, incompatible version of ns_return for example, it should be called something else and a compatibility wrapper written for ns_return that calls the new thing. In the best case, TCL compatibility should NEVER be broken. If we end up with a huge compatbility library, perhaps all written in TCL, that's much preferable to telling people to change their code. Breaking compatibility at the C level is more acceptable to me, because my guess is that very few people are using it. We have a small handful of C modules, so yeah, we may have to change them if the C API changes, but it would not be a big deal. Even if incompatible changes are allowed at major versions, what may very well happen is that it fractures the community and leaves some people running old versions longer than they want. We're in this situation now because of ns_shares: someone decided it was inconvenient to provide ns_share because of TCL core modifications, which I totally understand. But, we have a code base with over 800 ns_share statements, and probably 5x that many variable references. It will be a cold day in Hell before we rewrite all our code to use nsv arrays, just because someone decided that ns_shares are inconvenient to support. If ns_share ever goes away completely, we won't be upgrading, no matter how great a new version is. The other paradox with good, reliable software is that people don't have as much of a pressing need to upgrade it. So while a development group is releasing version after version of a product, and may have even released several production versions, there will be many customers that have not even tried the new version. Eventually, there will be a pressing need that forces the customer to consider upgrading. The Linux OS is an example of this, for us. We didn't upgrade to 2.4 until November of 2003, even though 2.4 had been released for a long time, because 2.2 was working fine for our needs. The 2 things that forced us to upgrade were that we needed to access more main memory, and our storage requirements increased to the point that we couldn't afford to do fsck's and needed a journaling file system. My point is, production customers may be running old versions much longer than developers think. If a customer has been happily running 3.4 AS for a few years and a compelling reason comes along for them to upgrade, and the current version of AS is 6.5, they don't want to run into a big incompatibility hurdle. If they run into too big of a hurdle, they will likely start considering all options, including moving to a different platform that they perceive is more stable from a compatibility point of view. Jim Zoran implemented some recent changes to TclX keyed lists in CVS HEAD which changed the C API, which broke at least some code at AOL which was still using the old C API. Rather than a wholesale back-out of the changes, I'd like to discuss two things: 1) Should changes that break backwards compatibility be allowed between minor revisions (i.e., 4.0 - 4.1) or should they be limited to major releases only (i.e., 4.x - 5.x). 2) Can we quickly implement some backwards compatibility for the TclX keyed list C API so that existing C code won't need to be modified/updated to use Zoran's new C API? What's the best way to do this? Can it be done through #define's? Or thin wrapper C procs that call the new C procs? Or, can we simply rename Zoran's new C procs to the old names to preserve compatibility? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Virtual hosting under 4.x and default Host: header
How do people feel about this? Is glob matching wanted enough to incur that per-request cost -- only if you're configured for software virtual hosting, of course. How about: try hash match if not found try glob matching against the list of virt. server wildcards if found a match use that server else use default server You only pay the glob penalty if the admin configures wildcard server names. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] SQLite
A suggestion: the SQLite DB driver could keep a serial number that is incremented whenever the DB is modified by the driver. The driver could then re-open the DB if the file handle it gets from gethandle is stale. It wouldn't work in the case of another process outside the web server modifying the database, but maybe that could be done with a periodic check of the DB file's timestamp. On Wed, Jul 14, 2004 at 02:51:44PM -0400, Dossy wrote: SQLite version 3.0 is made possible in part by AOL developers supporting and embracing great Open-Source Software. I wonder what this is all about. Anyone know? No idea. Sounds interesting, though. :) And, from the FAQ at http://www.sqlite.org/faq.html: | Note that if two or more threads have the same database open and | one thread creates a new table or index, the other threads might | not be able to see the new table right away. You might have to get | the other threads to close and reopen their connection to the | database before they will be able to see the new table. Pargraph #3 bothers me. If this is the case (changes in one thread not picked up by the other threads unless they close/reopen the DB), that's a pretty tough limitation. Dossy, like it or not, that's because by design, the ONLY operating system services SQLite uses for locking are a file locks on the database file. (More or less; if I remember right, it's actually a bit more complicated than that, but that's the general idea.) For AOLserver use, you basically have two simple options: 1. Open and close the SQLite db file for EVERY transaction. This is the normal way to use SQLite, and is definitely what you have to do if other processes might also be accessing the same database. 2. Keep the database open, and serialize all SQLite operations through a single thread. I would prefer to see an optional single-process many-threads MVCC concurrency model for SQLite, and several variants of that have been discussed rather extensively on the SQLite list. But so far no one has implemented it, and it's rather outside Dr. Hipp's primary design goals for SQLite. SQLite 3.0 introdues some new stuff to improve concurrency, but it doesn't go nearly as far as full-blown MVCC. That said, the SQLite wiki has numbers suggesting that the concurrency achievable even with the older 2.8.x SQLite is probably adequate for many small to medium websites: http://www.sqlite.org/cvstrac/wiki?p=WhenToUseSqlite My personal SWAG, is that if you can run your website ok on MySQL, you can probably run it just fine on SQLite too. -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] file upload corrupts jpg images
I seem to recall that the form file upload handler was rewritten in C for 3.5. It was in TCL before that. You could write a test script that does your file upload and runs an md5 on the temp file, while it still exists, with the exec command. Do a bunch of those with the old server and new server to see if there are any differences in the checksums. That will give you a test case and then you can start fiddling with things to find out why they are different, assuming that is the problem. Jim So, it appears that the file content is being rearranged. (on file upload w/3.5) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
From my point of view, focusing on protocols is not necessarily the only reason to want thread pools. I'm not all that familiar with AS 4 yet, so maybe I'm being dense here... keep that in mind. :) The reason thread pools would be interesting to me personally is to be able to control quality of service better. For example, I may want a pool of 5 threads to handle search engine spidering requests, and a pool of 5 threads to handle PHP requests, 5 threads to handle a particular application that uses a different database, 5 threads to handle CGI requests, 10 threads to TCL requests, 5 to handle image serving (fastpath), 5 to handle all other requests. I'd want to dispatch the requests to the various thread pools myself, using TCL code. The protocol and communication method (TCP vs UDP) part are all the same here - HTTP. Perhaps this can all be done today with AS 4 using multiple IP addresses - I dunno. However, sometimes it would be useful to have the pool functionality with one IP address. If a search engine decides to aggressively index your site, they have to do it on the main IP address. Being able to give them a different class of service, ie, a different thread pool, would be useful. We've already done a similar thing, by setting up multiple AS instances. But it would have been easier in many cases to set it up with thread pools. Much less code on our end to change. A way to accomplish the same thing today is by putting an intelligent switch/cache/proxy device in front of multiple servers, and having it route certain kinds of requests to certain servers. The problem from my point of view is: - the switch is another device to manage - it's an extra $5000+ - it's often a proprietary solution - if you look at client IP addresses, you have IP address issues to deal with, or have to set up some fancy routing, or have to modify your code to get the true peer address from a made-up header - you have to learn how to use the device's routing rules instead of making rules yourself, with TCL - hit logging will probably work differently; another difference to handle Another advantage to using thread pools in this application is that it may help a site be less vulnerable to DOS attacks. If a site is running 3rd party bulletin board software for example, and someone decides to run an attack against it, it may still kill that the board service/thread pool, but the other services running on the same server instance would not be unaffected. Without this partitioning, the attack might use all of the threads and kill the site. Maybe the thread pools could also have priorities too. So I could make dispatching decisions based on the current request load, like, If there are more than 5 threads active in the main pool, hold off executing any new requests in the bulletin board pool, ie, queue the request, but don't let it execute. In an ideal world, there would always be enough resource to do everything, but in the real world, there are sometimes momentary resource shortages that cause things to get slow. If something is going to be slow, I'd rather it be a non-essential service like a bulletin board rather than the threads generating our main pages. Is this something critical I can't live without? No. But it seems useful. Jim I definitely vote for a). I think AS has a very powerful framework that can be used outside of http. --- Zoran Vasiljevic [EMAIL PROTECTED] wrote: On Tuesday 17 August 2004 10:32, Zoran Vasiljevic wrote: Hi! I would like to start a discussion which will eventually lead to one of the two proposals listed below (one from Vlad Seryakov and the other from Stephen Deasey) included in the core-server distribution. Eh... wrong turn again... By start a discussion, I really expected the people to respond with: a. YES, I like AS to be more powerful multiprotocol server instead of being a powerful http-server only. And, I like proposal X better than Y because of Z. b. NO, I do not need AS doing anything else than http and am perfectly happy with status-quo now. Just forget the whole thing. I apologize for not being more precise. If a. turns out to be everbody's darling, then there are two excellent proposals for making this work. If somebody has a better solution, lets see this as RFE. Then, get a couple of (knowledgeable) people vote in democratic way about what's the better solution. I think there is a AOLSERVER-DEVEL (or such) list where deeper technical issues can be discussed in a smaller audience. If b. well, than it is clear. We just leave it as-is. Can we agree on this? Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver -
Re: [AOLSERVER] QOS
Hi - To put spiders in a separate thread pool, mapping requests to
thread pools by url doesn't work. You have to map by regular
expression on the User-Agent header or by IP address, assuming you can
figure out where Google's robots live. That's why I was suggesting
that the dispatching algorithm needs to be a registered proc.
Jim
The reason thread pools would be interesting to me personally is to be
able to control quality of service better. For example, I may want a
pool of 5 threads to handle search engine spidering requests, and a
pool of 5 threads to handle PHP requests, 5 threads to handle a
particular application that uses a different database, 5 threads to
handle CGI requests, 10 threads to TCL requests, 5 to handle image
serving (fastpath), 5 to handle all other requests. I'd want to
dispatch the requests to the various thread pools myself, using TCL
code. The protocol and communication method (TCP vs UDP) part are
all the same here - HTTP.
Perhaps this can all be done today with AS 4 using multiple IP
addresses - I dunno.
From sample-config.tcl ... you can map by URL, which means that by proper
partition of your URL-space you can do all you discuss above but shove
requests from spiders into their own thread pool.
#
# Example: Multiple connection thread pools.
#
# To enable:
#
# 1. Define one or more thread pools.
# 2. Configure pools as with the default server pool.
# 3. Map method/URL combinations to the pools
#
# All unmapped method/URL's will go to the default server pool.
#
#ns_section ns/server/server1/pools
#ns_section slow Slow requests here.
#ns_section fast Fast requests here.
#
#ns_section ns/server/server1/pool/slow
#ns_param map {POST /slowupload.adp}
#ns_param maxconnections 100 ;# Max connections to put on queue
#ns_param maxdropped 0 ;# Shut down if dropping too many conns
#ns_param maxthreads 20;# Tune this to scale your server
#ns_param minthreads 0 ;# Tune this to scale your server
#ns_param threadtimeout 120 ;# Idle threads die at this rate
#
#ns_section ns/server/server1/pool/fast
#ns_param map {GET /faststuff.adp}
#ns_param maxthreads 10
#
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Progress report
Hi - a few suggestions/ideas: 1. How many requests per second do you average, and what are your burst rates like? What are your vmstat numbers like during average and burst request processing, and how do they compare to when your system is lightly loaded? Getting familiar with these numbers will help you decide if you have a system capacity problem that perhaps only shows up during a traffic spike. 2. If increasing minthreads decreased the problems, it sounds like you may have trouble starting new threads during the day. Do you have a huge TCL library? This can significantly delay thread startup, because a very large TCL string gets eval'd to create your library procs. If you have a couple of threads running and a robot decides to spider your site, it may blast you with 10 requests. Your server needs to create 8 new threads, initialize them, handle these requests, and queue all new incoming requests. If a server is near its capacity, a delay of even a few seconds can take a while to recover from. Ex: your max processing rate is 10 Req/sec, your average actual request rate is 8 per second. You stop handling requests for 5 seconds, so 40 requests queue up. It'll take 20 seconds to clear this backlog, because you are near capacity. 3. How big are the pages you are returning? Do most of your pages fit into a socket buffer (usually 32K bytes)? If not, you'll tie up your connection threads spooling http output and your server will appear dead while incoming requests are queued. This is the main issue that prevents using a small maxthreads. Like others, I'd suggest you bump maxthreads up to 30 or so, but I'd also bump minthreads up to 30 because it sounds like you do have the thread startup problem I mentioned in #2. 4. I'd run a time-stamped vmstat trace throughout the day to make sure your system isn't just running out of steam. On Linux, disk I/O bottlenecks (specifically, flushing dirty buffers) can frequently cause a system to appear dead for several seconds, and recovering from this could take a while if you are already running near capacity. 5. If your system is truly idle when it is not responding to web requests, ie, vmstat shows no I/O activity and no CPU activity, you may have some kind of locking problem/bug. I don't know anything about ACS, so don't know if it extensively uses locks or not. Good luck! Jim To refresh everyone's memory, I have a situation where a couple of older ACS sites on the same system, running nsd3.3+ad13, started hanging multiple times per day. Most of the time by the time I receive notification from uptime, the problem has passed and leaves no clue behind. Sometimes I do manage to catch it still hung, but I don't know gdb well enough to know what to do, so I just restart it. Based on suggestions here, I set minthreads to 10 and threadtimeout to 3600. Maxthreads was already set to 10, so I left it alone. Since doing that there has been a significant reduction in the number of these incidents; one or less per day per site instead of every few hours. But less is not yet zero. I'm going to try experimenting with raising the above numbers some more, but anyone with a more educated guess than mine on what might be wrong would be welcome. I'd also appreciate some simple steps I could take to try to figure out what's wrong next time I catch the server in a hung state; I looked at the gdb doc Andrew posted a while back and although it would be a great reference if I knew what I was doing, it's not enough of a guide to tell me what I should be looking for. thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
maxconnections should be high, like 100. This is the number of keepalive socket connections your server maintains. These are low overhead, and make it faster for active browsers to get pages from you site since they don't have to reconnect. We use a keepalive timeout of 15 (seconds). That means roughly that 100 people can stay connected to your server at once, as long as they are making requests less than 15 seconds apart. If they go longer than that between requests, they lose their connection and have to reconnect. Jim On Aug 18, 2004, at 9:47 AM, Dossy Shiobara wrote: What class of hardware are these sites running on? maxthreads=10 is pretty low, IMHO. Running one site on a 1.2 GHz P3 running Linux, I'd comfortably set minthreads=maxthreads=30. BTW, what's the guidelines on setting maxconnections? Should it be the same as max and min threads? thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS
figure out where Google's robots live. That's why I was suggesting that the dispatching algorithm needs to be a registered proc. Perhaps it could be both. Yeah, that would be nice. No doubt many people would want to group just by URL, and that would solve several of the cases I raised. Being able to do this just with config commands would be good/easy. If combining the two methods, maybe the registered dispatcher proc only sees the unclaimed requests not already handled by config file dispatching. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Yep, there are always cases where any given solution won't work well. Using the HTTP protocol to download a a 600MB CD image or to implement a movie server, is clearly different than average web usage. To display the _directory_ of the CD image, or download individual files, output spooling would be a win, IMO. For fastpath (static file serving), reading a file into a memory buffer would seem to scale well for typical web size images: 250K or less, especially considering that fastpath already uses a memory cache. I can see 3 scenarios: 1. File is small: put it in the cache, leave it there after spooling 2. File exceeds cache size limit, but is less than 1MB: put it in the cache, delete it after spooling or make it expire very soon 3. File is huge: let the connection thread spool it, like it does now, and don't spool it, or My apologies for stomping on your topic. I do think the ability to handle other protocols is a good idea, esp for sites that need to integrate user registration with a lot of different packages. Jim I don't think the read-the-whole-file-in-the-conn-thread solution scales very well when faced with a directory of CD images or a distribution of Debian packages :-) ... On Thu, 2004-08-19 at 06:14, Jim Wilcoxson wrote: I guess I was unclear in my earlier note: 1. The HTTP driver thread reads all input for request, w/o blocking. My understanding is that this is now happening in AS 4 - great! 2. A conn service thread creates the output, either by running a TCL script, executing fastpath to read a file into memory, etc. This might do stats, file I/O, database I/O, etc., and may block. The result of this thread is a memory buffer. 3. The HTTP driver thread spools the memory buffer back to the client, w/o blocking. I wasn't suggesting that there should be no blocking in step 2, when the output is being created. That's why multiple connection service threads are needed. But the problem of a connection service thread being tied up while it spools a large page/image is eliminated. In the application of an image server, it would be nice if there could be one fastpath thread that did all non-blocking filesystem I/O (actually just reading is needed), or an asynchronous sendfile, but I agree with you that there are not good portable ways to do that yet. Jim ge. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] The 'weird 0 length problems' revisited
You can also get a zero content length logged if you have a C module that does Ns_ConnWrite and writes the headers and content itself, then closes the connection (even if there is a valid Content-Length header on the response), or if nothing is written out - for example, your TCL script goofs up and does a premature return before doing ns_return. Jim On 2004.12.08, Bernd Eidenschink [EMAIL PROTECTED] wrote: last year there was a thread concerning a 'weird 0 length problem': Wojciech Kocjan discovered requests served with a Content-Length of 0. As I see it there was neither a solution to that problem nor a bug to be found at that time. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver facelift.
I am a bit on the sidelines, because of a) time constraints, and b) we're using ns_shares heavily and so are stuck with TCL 7.x, which is no longer supported. (I know there are hacks in TCL 8 for ns_shares, but when we benchmarked them, they were much slower. So no compelling reason to upgrade, for us, at least not yet.) It seems that one of the main issues facing AS is a lack of people contributing their time to a group effort. People are doing things on their own, mainly for their own needs. It's understandable. Tweaking AS requires a high level of skill, and most of the people with that skill level have their hands full with their own projects. But it seems like having 2 separate versions will make cooperative efforts harder, not easier. JMHO. You know what I think would widen the appeal of AS? Optimize CGI and provide easy FastCGI support, or some other mechanism that makes it very fast and easy to run canned applications: bulletin board software, etc. Maybe AS CGI is already good enough to compete with Apache mod_perl, mod_php, ... I dunno. Or maybe it could never compete. But if it could, and there were tests/benchmarks to prove it, it seems that would go a long way to making AS more appealing to the general I need a good web server type of person. The recent discusion on the list about how to get an accounting application to run with mod_perl is a good example: the answer is you don't need to use mod_perl or anything like it. But I think there is confusion for people evaluating web server software, because mod_perl and mod_php are accepted as the best way of doing things - on Apache. We have an application that we've always exec'd; not through CGI, just exec. I converted it to a module because I wanted to save some state info. I expected a big performance increase too, just because it would be a module and not need to exec. But in the initial version, which was basically the same but implemented as a module instead of exec, there was hardly any performance difference. Modules are great for saving state and doing fancy stuff, but I think there is a general impression that anything good has to be done as an AS module, and that isn't true. Another thing I think would be good are some documented performance tests with well-known applications. For example, a benchmark comparing PHPBB under Apache and AS. Maybe Apache would beat the pants off AS - I dunno. Or we might be surprised. Or a whole suite of bulletin-board tests: vbulletin, phpbb, ultimatebb, etc., compared on several different web servers with the same hardware config. Published benchmarks are good because: a) they show that standard applications run b) they give people an idea of how AS compares to other options. Even if people aren't planning to run bbs software, they may be planning to run a large perl or php application, so knowing that other large applications work well gives people an extra comfort level. If it could be shown that AS runs some of these mainstream web applications better than Apache, the authors of the software would be likely to recommend AS as a solution on their site, thus increasing its popularity with little effort. Of course, if it turns out that AS performance sucked on these applications, then the goal of getting more people involved and making the server more appealing to the masses is misguided, and it would be better to focus on the smaller group of sites where deploying AS makes sense. Jim This *is* a suggestion for cooperative development of two versions in the interests of AOL and the community, perhaps with oversight by a steering committee if you like. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver facelift.
Hi - I read the entire thread mentioned below, and missed any mention of anyone saying the AS maintainers would not accept FastCGI. The other thing I guess I'm confused about, is that CGI is implemented as an AS module. Why can't someone write a FastCGI module, and whoever wants it can load it to get FCGI support? Does the core have to be modified to support FastCGI? And just as a my dog isn't in this fight observer, this discussion seems to be getting rather personal, which probably is not going to help it proceed productively. Jim Here's the kicker: The guy who'd probably do most of the work, John Sequeira, seems to think that option 1. is a better idea technically, but is leaning towards 2. instead, because after following the AOLserver multi-protocol discussions, he doubts that any FastCGI work would ever be accepted by the AOLserver maintainers, no matter what! http://openacs.org/forums/message-view?message%5fid=263944 That looks symptomatic of a problem in how the AOLserver codebase is managed to me. Dossy, doesn't it look that way to you too? -- Andrew Piskorski [EMAIL PROTECTED] http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver, NTLM, Apache+mod_proxy,
Hi - I don't know how others feel, but one of the community issues I'm having is that a mailing list is not a great place to have 5 discussions going on at once, IMO. If the mailing list does get more active (which in a way would be great), I would probably have to just unsubscribe. I think it's great that people are having in-depth discussions about how fast-cgi works, OACS, the issues with implementing it, etc., but since that isn't on my radar and I have zero interest in OACS, it's mostly noise to me. It'd be much easier IMO if how to get fastcgi on AS were a topic on a topic-based bulletin-board. THen people could look at things that were directly relevant or of interest. The other advantage of a topic-based board is that when new people join, they can easily get a sense of history. I know (or think) there is an archived copy of the mailing list, but doubt many people are going to go through 8-10 years of a mailing list. But, they might go poking through topics that look interesting on a board. To keep people in the loop, maybe a daily digest of topics with activity could be emailed. Just a thought... Jim On 2005.02.08, John Sequeira [EMAIL PROTECTED] wrote: When you say integrated security are you talking about the NTLM auth scheme for HTTP? As long as mod_proxy properly handles HTTP Keep-Alive, -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver Applications Spec DRAFT
Maybe an interface to memcached would make sense... Nsvs are, to me, global in essence. As such, I don't really see making namespace aware as a good goal. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
