Re: [AOLSERVER] Support for non-HTTP protocols
a. YES, I like AS to be more powerful multiprotocol server instead of being a powerful http-server only. And, I like proposal X better than Y because of Z. I vote for (a). I read both discussions from the sourceforge-links you provided and I see no jealousness or vanity between the two patch-providers, so I'm optimistic they will provide good information/arguments for a discussion with a final vote. Regards, Bernd. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS
Hi - To put spiders in a separate thread pool, mapping requests to
thread pools by url doesn't work. You have to map by regular
expression on the User-Agent header or by IP address, assuming you can
figure out where Google's robots live. That's why I was suggesting
that the dispatching algorithm needs to be a registered proc.
Jim
The reason thread pools would be interesting to me personally is to be
able to control quality of service better. For example, I may want a
pool of 5 threads to handle search engine spidering requests, and a
pool of 5 threads to handle PHP requests, 5 threads to handle a
particular application that uses a different database, 5 threads to
handle CGI requests, 10 threads to TCL requests, 5 to handle image
serving (fastpath), 5 to handle all other requests. I'd want to
dispatch the requests to the various thread pools myself, using TCL
code. The protocol and communication method (TCP vs UDP) part are
all the same here - HTTP.
Perhaps this can all be done today with AS 4 using multiple IP
addresses - I dunno.
From sample-config.tcl ... you can map by URL, which means that by proper
partition of your URL-space you can do all you discuss above but shove
requests from spiders into their own thread pool.
#
# Example: Multiple connection thread pools.
#
# To enable:
#
# 1. Define one or more thread pools.
# 2. Configure pools as with the default server pool.
# 3. Map method/URL combinations to the pools
#
# All unmapped method/URL's will go to the default server pool.
#
#ns_section ns/server/server1/pools
#ns_section slow Slow requests here.
#ns_section fast Fast requests here.
#
#ns_section ns/server/server1/pool/slow
#ns_param map {POST /slowupload.adp}
#ns_param maxconnections 100 ;# Max connections to put on queue
#ns_param maxdropped 0 ;# Shut down if dropping too many conns
#ns_param maxthreads 20;# Tune this to scale your server
#ns_param minthreads 0 ;# Tune this to scale your server
#ns_param threadtimeout 120 ;# Idle threads die at this rate
#
#ns_section ns/server/server1/pool/fast
#ns_param map {GET /faststuff.adp}
#ns_param maxthreads 10
#
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with
the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field
of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Progress report
On Aug 18, 2004, at 9:47 AM, Dossy Shiobara wrote: What class of hardware are these sites running on? maxthreads=10 is pretty low, IMHO. Running one site on a 1.2 GHz P3 running Linux, I'd comfortably set minthreads=maxthreads=30. BTW, what's the guidelines on setting maxconnections? Should it be the same as max and min threads? thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
Hi - a few suggestions/ideas: 1. How many requests per second do you average, and what are your burst rates like? What are your vmstat numbers like during average and burst request processing, and how do they compare to when your system is lightly loaded? Getting familiar with these numbers will help you decide if you have a system capacity problem that perhaps only shows up during a traffic spike. 2. If increasing minthreads decreased the problems, it sounds like you may have trouble starting new threads during the day. Do you have a huge TCL library? This can significantly delay thread startup, because a very large TCL string gets eval'd to create your library procs. If you have a couple of threads running and a robot decides to spider your site, it may blast you with 10 requests. Your server needs to create 8 new threads, initialize them, handle these requests, and queue all new incoming requests. If a server is near its capacity, a delay of even a few seconds can take a while to recover from. Ex: your max processing rate is 10 Req/sec, your average actual request rate is 8 per second. You stop handling requests for 5 seconds, so 40 requests queue up. It'll take 20 seconds to clear this backlog, because you are near capacity. 3. How big are the pages you are returning? Do most of your pages fit into a socket buffer (usually 32K bytes)? If not, you'll tie up your connection threads spooling http output and your server will appear dead while incoming requests are queued. This is the main issue that prevents using a small maxthreads. Like others, I'd suggest you bump maxthreads up to 30 or so, but I'd also bump minthreads up to 30 because it sounds like you do have the thread startup problem I mentioned in #2. 4. I'd run a time-stamped vmstat trace throughout the day to make sure your system isn't just running out of steam. On Linux, disk I/O bottlenecks (specifically, flushing dirty buffers) can frequently cause a system to appear dead for several seconds, and recovering from this could take a while if you are already running near capacity. 5. If your system is truly idle when it is not responding to web requests, ie, vmstat shows no I/O activity and no CPU activity, you may have some kind of locking problem/bug. I don't know anything about ACS, so don't know if it extensively uses locks or not. Good luck! Jim To refresh everyone's memory, I have a situation where a couple of older ACS sites on the same system, running nsd3.3+ad13, started hanging multiple times per day. Most of the time by the time I receive notification from uptime, the problem has passed and leaves no clue behind. Sometimes I do manage to catch it still hung, but I don't know gdb well enough to know what to do, so I just restart it. Based on suggestions here, I set minthreads to 10 and threadtimeout to 3600. Maxthreads was already set to 10, so I left it alone. Since doing that there has been a significant reduction in the number of these incidents; one or less per day per site instead of every few hours. But less is not yet zero. I'm going to try experimenting with raising the above numbers some more, but anyone with a more educated guess than mine on what might be wrong would be welcome. I'd also appreciate some simple steps I could take to try to figure out what's wrong next time I catch the server in a hung state; I looked at the gdb doc Andrew posted a while back and although it would be a great reference if I knew what I was doing, it's not enough of a guide to tell me what I should be looking for. thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Given the current movement of this thread in the direction of You're wrong... no, you are!!, I figured I'd toss something out in an attempt to set it back on track a bit. What, exactly, is the goal that we are trying to accomplish by applying one of these patches? Can we put together a list of goals and a list of constraints, and then analyze whether or not these patches meet those requirements? If they don't, we can then go ahead and determine what is missing, and if its something that can be added later, or it the approach of the patch would preclude it. My thoughts, which are only partially complete, would include: Requirements: - The ability to add handlers for additional protocols - The ability to do so easily, at a very high level, and - The ability to put forth more effort, but access things at a low level (most likely for the purpose of performance) - The high level API for writing a driver should allow the programmer to write simple network code, and let all the logging, connection pooling, thread pooling, etc. be handled automatically. - The low level API should allow the programmer to handle every aspect of the code independently, including handing off requests to different threads, etc. - The ability to write protocol handlers in Tcl, and move them to C if needed for performance. Constraints: - Must be decoupled from the HTTP driver, so that that driver can be changed or replaced at any time without impacting any added drivers Notes: - I'd like to see the Handler API be sufficient at the low level to allow an HTTP handler to be written that is every bit as efficient and speedy as what is being worked on for 4.1 - I'd like to see the high level API be easy enough to use that one could register an echo server handler and write the code for it in a dozen lines or so - I'd like the progression from high level to low level be gradual, so the programmer can address only those issues that pertain to his specific protocol Rob Seeger -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
maxconnections should be high, like 100. This is the number of keepalive socket connections your server maintains. These are low overhead, and make it faster for active browsers to get pages from you site since they don't have to reconnect. We use a keepalive timeout of 15 (seconds). That means roughly that 100 people can stay connected to your server at once, as long as they are making requests less than 15 seconds apart. If they go longer than that between requests, they lose their connection and have to reconnect. Jim On Aug 18, 2004, at 9:47 AM, Dossy Shiobara wrote: What class of hardware are these sites running on? maxthreads=10 is pretty low, IMHO. Running one site on a 1.2 GHz P3 running Linux, I'd comfortably set minthreads=maxthreads=30. BTW, what's the guidelines on setting maxconnections? Should it be the same as max and min threads? thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
On 2004.08.18, Janine Sisk [EMAIL PROTECTED] wrote: On Aug 18, 2004, at 9:47 AM, Dossy Shiobara wrote: What class of hardware are these sites running on? maxthreads=10 is pretty low, IMHO. Running one site on a 1.2 GHz P3 running Linux, I'd comfortably set minthreads=maxthreads=30. BTW, what's the guidelines on setting maxconnections? Should it be the same as max and min threads? There's no hard rules, but I'd recommend setting maxconns up around 100-150. Essentially, 30x-50x the threads, as a rough guide. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Wednesday 18 August 2004 16:43, Robert Seeger wrote: Given the current movement of this thread in the direction of You're wrong... no, you are!!, I figured I'd toss something out in an attempt to set it back on track a bit. What, exactly, is the goal that we are trying to accomplish by applying one of these patches? Can we put together a list of goals and a list of constraints, and then analyze whether or not these patches meet those requirements? If they don't, we can then go ahead and determine what is missing, and if its something that can be added later, or it the approach of the patch would preclude it. THANK YOU! This is exactly what WAS the intention of the discussion. The goal/intention in a nutshell: Allow AS to be a fast and scalable any_protocol instead of the http_only protocol server. This should *not* harm AS main role of being the good http server. It should *augment* its capabilities, rather. If you look in the past: In 3.x there was Win32 support. In 4.0 it was taken out. Luckily, Jamie Rassmusen, (thank's Jamie) went and got the support back into the core. In 3.x it was possible to serve alternative protocols. In 4.0 the hook was taken out. Why? Because AOL could not maintain it? As with Win32? But there are people in the community who *have* the knowledge and time to do it. And more importantly: they need it, otherwise they would not go that far to write complete ready-to-use code including sample usage and post it to SF as RFE's. Before doing anything else, I would kindly ask anybody interested to go and *look* at the provided patches and build his/her own oppinion. By doing this, most of the questions will immediately be answered, or for the ones which didn't, the patch authors will be happy (I assume) to give any explanations needed. They tried so, numerous times, as evident from the responses so far. I think accepting the idea is much more important then the actual given implementation. I can't really see how this might, even remotely, harm the project. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS
figure out where Google's robots live. That's why I was suggesting that the dispatching algorithm needs to be a registered proc. Perhaps it could be both. Yeah, that would be nice. No doubt many people would want to group just by URL, and that would solve several of the cases I raised. Being able to do this just with config commands would be good/easy. If combining the two methods, maybe the registered dispatcher proc only sees the unclaimed requests not already handled by config file dispatching. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
On Wed, 2004-08-18 at 09:22 -0400, Janine Sisk wrote: Based on suggestions here, I set minthreads to 10 and threadtimeout to 3600. Maxthreads was already set to 10, so I left it alone. You mention OpenACS -- I had a server that got bogged down tremendously the other day when it got hit with about 2100 HEAD requests from script kiddies looking for exploits. It seems that a HEAD request is still processed by OpenACS, and even though it is a HEAD request, it does set up a session and set a cookie. Have you checked your weblogs around the downtime periods to see what other action might be taking place? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
On Aug 18, 2004, at 1:09 PM, Chris Davies wrote: It seems that a HEAD request is still processed by OpenACS, and even though it is a HEAD request, it does set up a session and set a cookie. As an FYI, the HTTP RFC basically states that servers should do all the processing on a HEAD request that would happen for a GET request; this ensures that the returned headers are accurate. See RFC 2616, Section 9.4 (although there's not much more there than what I just mentioned). Noah Robin System Administrator, America Online 703.265.2925 #include remarks/witty.h -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
expected behavior yes. Actual behavior, no telnet aol.com 80 Trying 205.188.145.213... Connected to aolr-v1.evip.aol.com. Escape character is '^]'. HEAD / HTTP/1.0 HTTP/1.0 404 Not Found MIME-Version: 1.0 Date: Wed, 18 Aug 2004 19:04:18 GMT Server: AOLserver/3.4.2 Content-Type: text/html Content-Length: 547 Connection: close telnet aol.com 80 Trying 205.188.145.213... Connected to aolr-v1.evip.aol.com. Escape character is '^]'. HEAD / HTTP/1.0 Host: aol.com HTTP/1.0 404 Not Found MIME-Version: 1.0 Date: Wed, 18 Aug 2004 19:04:59 GMT Server: AOLserver/3.4.2 Content-Type: text/html Content-Length: 547 Connection: close Connection closed by foreign host. Not everyone follows the RFCs. :) On Wed, 2004-08-18 at 13:55 -0400, Noah Robin wrote: On Aug 18, 2004, at 1:09 PM, Chris Davies wrote: It seems that a HEAD request is still processed by OpenACS, and even though it is a HEAD request, it does set up a session and set a cookie. As an FYI, the HTTP RFC basically states that servers should do all the processing on a HEAD request that would happen for a GET request; this ensures that the returned headers are accurate. See RFC 2616, Section 9.4 (although there's not much more there than what I just mentioned). -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress report
On 2004.08.18, Michael Matthews [EMAIL PROTECTED] wrote: On Aug 18, 2004, at 3:08 PM, Chris Davies wrote: expected behavior yes. Actual behavior, no telnet aol.com 80 That would be because you're going to a redirect farm, not the actual site. Try www.aol.com. That's not the issue. See: $ telnet aol.com 80 Trying 205.188.145.213... Connected to aolr-v1.evip.aol.com. Escape character is '^]'. HEAD / HTTP/1.0 Host: aol.com:80 HTTP/1.0 404 Not Found MIME-Version: 1.0 Date: Wed, 18 Aug 2004 20:12:19 GMT Server: AOLserver/3.4.2 Content-Type: text/html Content-Length: 547 Connection: close Connection closed by foreign host. $ telnet aol.com 80 Trying 205.188.145.213... Connected to aolr-v1.evip.aol.com. Escape character is '^]'. GET / HTTP/1.0 Host: aol.com:80 HTTP/1.0 302 Found Location: http://www.aol.com/ MIME-Version: 1.0 Date: Wed, 18 Aug 2004 20:12:26 GMT Server: AOLserver/3.4.2 Content-Type: text/html Content-Length: 291 Connection: close !DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN HTML HEAD TITLERedirection/TITLE /HEAD BODY H2Redirection/H2 A HREF=http://www.aol.com/;The requested URL has moved here./A P ALIGN=RIGHTSMALLIAOLserver/3.4.2 on http://www.aol.com/I/SMALL/P /BODY/HTML The HEAD request returns 404, but the GET request returns a 302 redirect. Something's wrong ... is the redirect implemented as a registered proc? Do registered procs not get invoked on HEAD requests? If so, that's a bug ... If this were the VIP fabricating an HTTP response, then why would it be adding the Server: AOLserver/3.4.2 header -- that's got to be coming from the origin HTTP server behind the VIP. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] AOLserver Project CVS Commit Guidelines
After the recent discussions, I felt it may be useful to codify what I see as the current CVS commit guidelines for project developers. The current text is up on the wiki: http://panoptic.com/wiki/aolserver/CVSCommitGuidelines I make reference to a Vision Statement which I am still in the process of drafting up. I'm hoping to have it ready for review sometime next week. I'm pasting the text here for the CVS Commit Guidelines but without formatting, markup or links: CVS Commit Guidelines In order to achieve our Vision Statement, the AOLserver Project solicits contributions from any interested party. There are a limited number of [AOLserver Project Members] for the AOLserver Project hosted at SourceForge who have access to accept and commit these contributions, and this document outlines the general policies governing this activity. 1. Everyone who has CVS commit access, has CVS commit access. Obviously true, but what does this really mean? It means that if you were given CVS commit access, most likely it was intentional and the interpretation of that action should be obvious: it has been acknowledged that the project wishes you to commit changes to the project. 2. Who grants CVS commit access? While there are currently a number of Project Administrators, any of whom technically can grant others CVS commit access, the current Project Owner should be the only person to grant new members CVS commit access. There is no guidelines for deciding who to grant and not grant such access; the decision is solely left to the discretion of the current Project Owner. 3. I have CVS commit access and someone has contributed a patch which fixes a bug. Should I just go and merge it in and commit? This is probably the hardest question to answer. Use the answer here as only a limited guideline to help you decide, and not as the authoritative rules governing this decision. In the case of a patch which fixes a bug, ensure that the bug is documented by being filed in the [SourceForge Bug Tracker] first. This gives others a chance to investigate the bug and comment on it. The contributed patch may fix the bug, but there may be other solutions that are better and should be considered. Sometimes, the fix is trivial: it's still useful to have the bug documented for future review of past changes and bug fixes. It's important that changes do not simply get merged in and committed without testing. If you wish to commit a change, do not do so unless you're also willing to commit the necessary time and resources necessary to test it. Sometimes, this just isn't feasible: use your best judgement, but it should generally be an exception and not the rule that changes get committed untested. Finally, if you are a Project Admin., assign the bug to yourself to indicate that you're taking ownership of it. Otherwise, append a comment indicating that you wish to do so and request the bug be assigned to you. Once you are owner of the bug, proceed with merging in the change and testing to ensure the bug has been fixed. (Once a regression test suite has been developed, ensure all regression tests pass as well.) After committing the change, append a note to the bug indicating that you did apply the change and that you committed it. 4. What about feature enhancement patches? I think the feature is desirable, can I commit the change? Like #3 above, this is only a suggestion and ultimately the decision should be made using your best judgement. However, here are some guidelines you may want to consider: * Every line of code added increases the overall maintenance burden of AOLserver. While perceived to be small in some cases, there is a non-zero cost, however small it might be. Just because a feature has value does not mean it's valuable enough to be added to AOLserver. * A feature that seems easy to implement today may make adding other new features or changing existing features more difficult tomorrow. * Once a feature is implemented a particular way and people begin to use it, especially once they've deployed it into a production environment, it is very important to carefully maintain backwards compatibility. An inadequately thought out implementation may result in the need to break backwards compatibility. It is worth deferring adding such a feature until the design has clear signs of longevity rather than adding it in an incomplete state and then having people rely on it. * Every new feature is an opportunity to add new bugs. Is this feature worth the risk? * Does this feature help bring AOLserver closer to meeting its intended goals from the Vision Statement? * Am I responsible for defending my decision to commit this change with my peers? If the feature turns out indeed have negative impact, do I feel comfortable with the responsibility for causing it? Do I have the
Re: [AOLSERVER] Support for non-HTTP protocols
Jim Wilcoxson wrote: IMO it is not scalable to use connection servicing threads to spool output when the size of the output exceeds the socket buffer capacity. Doing I/O with non-blocking, event-driven threads also helps prevent DOS attacks. The trade-off is in memory usage: when a connection service thread does an ns_return, and the output exceeds the socket buffer size, you need to hang onto the buffer in memory somewhere while it is spooled by the HTTP I/O driver. The advantage is that the connection service thread can be off handling another request, but there is also a new resource to manage: spooled output buffers. For returning files, you can always work the way [fcopy] does - leave the fp open and when the socket is writable, read some more and send it. This wouldn't be too resource consuming. I've never seen too many HTML files above 100K (20s on a 57k6 modem anyone? :), usually the largest ones are static files. On average, my dynamic HTML code is usually 20-30K at most. The problem could be with people storing files as BLOBs, but then again they have to be read into memory and sent back to the client now as well... So this doesn't change much. -- WK -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
