Re: [AOLSERVER] Support for non-HTTP protocols
A portable way to handle really huge files would be to let the conn service threads (I'm thinking primarily of fastpath) co-routine with the HTTP driver: HTTP socket driver reads all of the request input Initial request is dispatched to fastpath: fastpath opens the file if new request stat file do ifmod set headers, etc. if file size is "small" add file to cache, return to output socket driver else if file size is "large", but < max cache entry size add file to cache, set quick expiration, return to output socket driver else if file size is > max cache entry size read large-sized hunk (250KB, or some configurable chunk size) set file offsets in conn structure & "incomplete" flag set quick expiration return to output socket driver fi else (continuation of incomplete request) position to last offset read a large hunk update file offsets in conn structure & set/clear "incomplete" flag return to output socket fi close file IMO, with this kind of setup, AOLServer could serve a thousand movie downloads or CD image downloads in an efficient manner, using just a handful of connection service threads and 250MB of memory (1000 x the max chunk size) for the output buffers. As a side benefit, it conserves (disk) file descriptors in a big way, because they are not in use while the data is being spooled to the socket. I'm not saying AS should be designed/optimized for serving movies or CD images, but IMO this setup would also make it a better (more scalable) image server, and that is an important webserver feature. I'm continuing to post on this subject because if changes are being considered to the design of the socket drivers to support multiple protocols, maybe it's a good time to consider output spooling too. To me, tuning the cache is a totally separate issue, but may also be useful. Jim > > On 2004.08.19, Jim Wilcoxson <[EMAIL PROTECTED]> wrote: > > For fastpath (static file serving), reading a file into a memory > > buffer would seem to scale well for typical web size images: 250K or > > less, especially considering that fastpath already uses a memory > > cache. I can see 3 scenarios: > > > > 1. File is small: put it in the cache, leave it there after spooling > > 2. File exceeds cache size limit, but is less than 1MB: put it in the > > cache, delete it after spooling or make it expire "very soon" > > 3. File is huge: let the connection thread spool it, like it does now, > > and don't spool it, or > > The Netscape Enterprise Server (NES) web server has a static file cache > that's tunable ... "nfsc" rings a bell. You specify things like the > max number of entries in the cache, the byte size ranges that define > "small", "medium" and "large" files, etc. > > Could be an interesting idea to accelerate fastpath static content > serving from AOLserver. Maybe. > > -- Dossy > > -- > Dossy Shiobara mail: [EMAIL PROTECTED] > Panoptic Computer Network web: http://www.panoptic.com/ > "He realized the fastest way to change is to laugh at your own > folly -- then you can let go and quickly move on." (p. 70) > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.19, Jim Wilcoxson <[EMAIL PROTECTED]> wrote: > For fastpath (static file serving), reading a file into a memory > buffer would seem to scale well for typical web size images: 250K or > less, especially considering that fastpath already uses a memory > cache. I can see 3 scenarios: > > 1. File is small: put it in the cache, leave it there after spooling > 2. File exceeds cache size limit, but is less than 1MB: put it in the > cache, delete it after spooling or make it expire "very soon" > 3. File is huge: let the connection thread spool it, like it does now, > and don't spool it, or The Netscape Enterprise Server (NES) web server has a static file cache that's tunable ... "nfsc" rings a bell. You specify things like the max number of entries in the cache, the byte size ranges that define "small", "medium" and "large" files, etc. Could be an interesting idea to accelerate fastpath static content serving from AOLserver. Maybe. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Yep, there are always cases where any given solution won't work well. Using the HTTP protocol to download a a 600MB CD image or to implement a movie server, is clearly different than average web usage. To display the _directory_ of the CD image, or download individual files, output spooling would be a win, IMO. For fastpath (static file serving), reading a file into a memory buffer would seem to scale well for typical web size images: 250K or less, especially considering that fastpath already uses a memory cache. I can see 3 scenarios: 1. File is small: put it in the cache, leave it there after spooling 2. File exceeds cache size limit, but is less than 1MB: put it in the cache, delete it after spooling or make it expire "very soon" 3. File is huge: let the connection thread spool it, like it does now, and don't spool it, or My apologies for stomping on your topic. I do think the ability to handle other protocols is a good idea, esp for sites that need to integrate user registration with a lot of different packages. Jim > I don't think the read-the-whole-file-in-the-conn-thread solution scales > very well when faced with a directory of CD images or a distribution of > Debian packages :-) ... > > > On Thu, 2004-08-19 at 06:14, Jim Wilcoxson wrote: > > I guess I was unclear in my earlier note: > > > > 1. The HTTP driver thread reads all input for request, w/o blocking. > > My understanding is that this is now happening in AS 4 - great! > > > > 2. A conn service thread creates the output, either by running a TCL > > script, executing fastpath to read a file into memory, etc. This > > might do stats, file I/O, database I/O, etc., and may block. The > > result of this thread is a memory buffer. > > > > 3. The HTTP driver thread spools the memory buffer back to the client, > > w/o blocking. > > > > I wasn't suggesting that there should be no blocking in step 2, when > > the output is being created. That's why multiple connection service > > threads are needed. But the problem of a connection service thread > > being tied up while it spools a large page/image is eliminated. > > > > In the application of an image server, it would be nice if there could > > be one fastpath thread that did all non-blocking filesystem I/O > > (actually just reading is needed), or an asynchronous sendfile, but I > > agree with you that there are not good portable ways to do that yet. > > > > Jim > > ge. You can leave the Subject: field of your email blank. > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
The difference between blocking disk I/O and other sources of conn thread 'blocking' is that the I/O problem can be solved with a small number of light weight threads which take the blocking on the chin for you, but there's not much you can do about arbitrary script processing. I don't think the read-the-whole-file-in-the-conn-thread solution scales very well when faced with a directory of CD images or a distribution of Debian packages :-) Dragging this back to the topic at hand, what this highlights to me is that infrastructure can be tricky, so you don't want to be re-implementing it ad-hoc. The Ns_ParseProc interface allows new protocol implementations to take full advantage of AOLserver's cool infrastructure, transparently. Cheers. On Thu, 2004-08-19 at 06:14, Jim Wilcoxson wrote: > I guess I was unclear in my earlier note: > > 1. The HTTP driver thread reads all input for request, w/o blocking. > My understanding is that this is now happening in AS 4 - great! > > 2. A conn service thread creates the output, either by running a TCL > script, executing fastpath to read a file into memory, etc. This > might do stats, file I/O, database I/O, etc., and may block. The > result of this thread is a memory buffer. > > 3. The HTTP driver thread spools the memory buffer back to the client, > w/o blocking. > > I wasn't suggesting that there should be no blocking in step 2, when > the output is being created. That's why multiple connection service > threads are needed. But the problem of a connection service thread > being tied up while it spools a large page/image is eliminated. > > In the application of an image server, it would be nice if there could > be one fastpath thread that did all non-blocking filesystem I/O > (actually just reading is needed), or an asynchronous sendfile, but I > agree with you that there are not good portable ways to do that yet. > > Jim > ge. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
I guess I was unclear in my earlier note: 1. The HTTP driver thread reads all input for request, w/o blocking. My understanding is that this is now happening in AS 4 - great! 2. A conn service thread creates the output, either by running a TCL script, executing fastpath to read a file into memory, etc. This might do stats, file I/O, database I/O, etc., and may block. The result of this thread is a memory buffer. 3. The HTTP driver thread spools the memory buffer back to the client, w/o blocking. I wasn't suggesting that there should be no blocking in step 2, when the output is being created. That's why multiple connection service threads are needed. But the problem of a connection service thread being tied up while it spools a large page/image is eliminated. In the application of an image server, it would be nice if there could be one fastpath thread that did all non-blocking filesystem I/O (actually just reading is needed), or an asynchronous sendfile, but I agree with you that there are not good portable ways to do that yet. Jim > On Wed, 2004-08-18 at 09:21, Jim Wilcoxson wrote: > > ... > > > > AOLserver is ideally suited to the majority of server tasks. About the > > > > only shortcoming which comes to mind is that conn threads are required > > > > to do blocking writes. But fixing that would be of benefit to the HTTP > > > > processing side too. > > > > > > So, we want reader threads and separate writer threads now, too? > > > Exactly how many free-range threads do we want roaming the server > > > prairie, here? > > > > You don't need reader threads and writer threads. If input and output > > both use event-driven, non-blocking I/O, you can use one thread to do > > both, or one thread per CPU, like TUX. > > > You can't do non blocking I/O on a file descriptor backed by disk. Not > portably at least. Calls like stat(2) to get the last mod time may also > block. > > That's why I suggested a pool of writer threads. TUX actually does > something very similar to handle potentially blocking disk I/O. > > > One of the biggest resource hogs is not the conn threads themselves but > the Tcl interps they drag round with them. If you're interested in > cutting down on resource usage, you might want to take a look at this > RFE: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1012103&group_id=3152&atid=353152 > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Wed, 2004-08-18 at 08:43, Robert Seeger wrote:
>
> ...
>
> Constraints:
> - Must be decoupled from the HTTP driver, so that that driver can be
> changed or replaced at any time without impacting any added drivers
I don't think my patch imposes a maintenance burden. It inserts the
following piece of code into the fastpath of the driver thread at the
end of driver.c:SockRead:
if (drvPtr->parseProc == NULL) {
return SockParse(connPtr);
}
i.e. if an alternative driver has not been registered, just carry on as
normal and parse the buffer of data expecting an HTTP request. This is
pretty trivial.
What follows it is a couple of lines of setup code which makes the
just-received buffer available to the Ns_ConnGets, Ns_ConnRead,
Ns_ConnReadLine, Ns_ConnContent etc. APIs.
The parseProc which implements the alternate protocol is then called.
Here's it's signature:
int (Ns_ParseProc) (void *arg, Ns_Conn *conn);
Pretty simple. This is the interface we're committing to support into
the future. I don't think I'm tying any body's hands here.
The first arg is just module specific callback data, the second a
pointer to the widely used Ns_Conn structure. That's not going away
anytime soon.
The return code can be one of: NS_SOCK_READY, NS_SOCK_MORE,
NS_SOCK_ERROR. Even if the internals of the driver thread were to
change radically, I don't think it would be too hard to continue
supporting this. Conversely, I don't think it would be too hard for
module writers to use an alternate API, considering it's small
footprint. Bigger changes have been made in the past.
And that's about it. The rest of the patch is trivial support code.
> Notes:
> - I'd like to see the Handler API be sufficient at the low level to
> allow an HTTP handler to be written that is every bit as efficient and
> speedy as what is being worked on for 4.1
Cut 'n paste the ~100 lines of code from SockParse which already
conforms to the Ns_ParseProc API and you're basically done.
> - I'd like to see the high level API be easy enough to use that one
> could register an "echo server" handler and write the code for it in a
> dozen lines or so
The example POP3 server, which is a simple, single-line oriented
protocol, implements an Ns_ParseProc in 24 line of code.
> - I'd like the progression from high level to low level be gradual, so
> the programmer can address only those issues that pertain to his
> specific protocol
A Tcl API could be wrapped around the Ns_ParseProc interface in the same
manner that ns_register_filter wraps Ns_RegisterFilter(). You would
have to make an interp available to the driver thread which doesn't
normally have one.
The Tcl code which implements your parser could call [ns_conn content]
and use the sls API to maintain state. Or you could wrap the
Ns_ConnReadLine etc. APIs.
Looks like an afternoons work.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Wed, 2004-08-18 at 09:21, Jim Wilcoxson wrote: > ... > > > AOLserver is ideally suited to the majority of server tasks. About the > > > only shortcoming which comes to mind is that conn threads are required > > > to do blocking writes. But fixing that would be of benefit to the HTTP > > > processing side too. > > > > So, we want reader threads and separate writer threads now, too? > > Exactly how many free-range threads do we want roaming the server > > prairie, here? > > You don't need reader threads and writer threads. If input and output > both use event-driven, non-blocking I/O, you can use one thread to do > both, or one thread per CPU, like TUX. You can't do non blocking I/O on a file descriptor backed by disk. Not portably at least. Calls like stat(2) to get the last mod time may also block. That's why I suggested a pool of writer threads. TUX actually does something very similar to handle potentially blocking disk I/O. One of the biggest resource hogs is not the conn threads themselves but the Tcl interps they drag round with them. If you're interested in cutting down on resource usage, you might want to take a look at this RFE: http://sourceforge.net/tracker/index.php?func=detail&aid=1012103&group_id=3152&atid=353152 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.19, Wojciech Kocjan <[EMAIL PROTECTED]> wrote: > For returning files, you can always work the way [fcopy] does - leave > the fp open and when the socket is writable, read some more and send it. > > This wouldn't be too resource consuming. I've never seen too many HTML > files above 100K (20s on a 57k6 modem anyone? :), usually the largest > ones are static files. On average, my dynamic HTML code is usually > 20-30K at most. Think: WebDAV. It's a real possibility that folks will upload and/or download files that exceed 10MB. > The problem could be with people storing files as BLOBs, but then again > they have to be read into memory and sent back to the client now as > well... So this doesn't change much. On Linux at least, use of sendfile() could be a win as the data doesn't have to go through userland before it gets shoveled out the socket back to the client. Probably not worth using given the many limitations on sendfile() though ... -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Jim Wilcoxson wrote: IMO it is not scalable to use connection servicing threads to spool output when the size of the output exceeds the socket buffer capacity. Doing I/O with non-blocking, event-driven threads also helps prevent DOS attacks. The trade-off is in memory usage: when a connection service thread does an ns_return, and the output exceeds the socket buffer size, you need to hang onto the buffer in memory somewhere while it is spooled by the HTTP I/O driver. The advantage is that the connection service thread can be off handling another request, but there is also a new resource to manage: spooled output buffers. For returning files, you can always work the way [fcopy] does - leave the fp open and when the socket is writable, read some more and send it. This wouldn't be too resource consuming. I've never seen too many HTML files above 100K (20s on a 57k6 modem anyone? :), usually the largest ones are static files. On average, my dynamic HTML code is usually 20-30K at most. The problem could be with people storing files as BLOBs, but then again they have to be read into memory and sent back to the client now as well... So this doesn't change much. -- WK -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Wednesday 18 August 2004 16:43, Robert Seeger wrote: > Given the current movement of this thread in the direction of "You're > wrong... no, you are!!", I figured I'd toss something out in an attempt > to set it back on track a bit. > > What, exactly, is the goal that we are trying to accomplish by applying > one of these patches? Can we put together a list of goals and a list of > constraints, and then analyze whether or not these patches meet those > requirements? If they don't, we can then go ahead and determine what is > missing, and if its something that can be added later, or it the > approach of the patch would preclude it. THANK YOU! This is exactly what WAS the intention of the discussion. The goal/intention in a nutshell: Allow AS to be a fast and scalable any_protocol instead of the http_only protocol server. This should *not* harm AS main role of being the good http server. It should *augment* its capabilities, rather. If you look in the past: In 3.x there was Win32 support. In 4.0 it was taken out. Luckily, Jamie Rassmusen, (thank's Jamie) went and got the support back into the core. In 3.x it was possible to serve alternative protocols. In 4.0 the hook was taken out. Why? Because AOL could not maintain it? As with Win32? But there are people in the community who *have* the knowledge and time to do it. And more importantly: they need it, otherwise they would not go that far to write complete ready-to-use code including sample usage and post it to SF as RFE's. Before doing anything else, I would kindly ask anybody interested to go and *look* at the provided patches and build his/her own oppinion. By doing this, most of the questions will immediately be answered, or for the ones which didn't, the patch authors will be happy (I assume) to give any explanations needed. They tried so, numerous times, as evident from the responses so far. I think accepting the "idea" is much more important then the actual given implementation. I can't really see how this might, even remotely, harm the project. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
... > > AOLserver is ideally suited to the majority of server tasks. About the > > only shortcoming which comes to mind is that conn threads are required > > to do blocking writes. But fixing that would be of benefit to the HTTP > > processing side too. > > So, we want reader threads and separate writer threads now, too? > Exactly how many free-range threads do we want roaming the server > prairie, here? You don't need reader threads and writer threads. If input and output both use event-driven, non-blocking I/O, you can use one thread to do both, or one thread per CPU, like TUX. IMO it is not scalable to use connection servicing threads to spool output when the size of the output exceeds the socket buffer capacity. Doing I/O with non-blocking, event-driven threads also helps prevent DOS attacks. The trade-off is in memory usage: when a connection service thread does an ns_return, and the output exceeds the socket buffer size, you need to hang onto the buffer in memory somewhere while it is spooled by the HTTP I/O driver. The advantage is that the connection service thread can be off handling another request, but there is also a new resource to manage: spooled output buffers. Threads are usually a more scarce resource on a server compared to memory and socket connections, so IMO it's better to use the more plentiful resources to handle load/capacity issues. On a busy server, it would be easy to store 1000 spooled output buffers, each one being 100K. That's only 100MB of memory, which is nothing for modern high-end systems. However, running 1000 connection threads to service this same load doesn't seem as reasonable to me. Jim -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Given the current movement of this thread in the direction of "You're wrong... no, you are!!", I figured I'd toss something out in an attempt to set it back on track a bit. What, exactly, is the goal that we are trying to accomplish by applying one of these patches? Can we put together a list of goals and a list of constraints, and then analyze whether or not these patches meet those requirements? If they don't, we can then go ahead and determine what is missing, and if its something that can be added later, or it the approach of the patch would preclude it. My thoughts, which are only partially complete, would include: Requirements: - The ability to add handlers for additional protocols - The ability to do so easily, at a very high level, and - The ability to put forth more effort, but access things at a low level (most likely for the purpose of performance) - The high level API for writing a driver should allow the programmer to write simple network code, and let all the logging, connection pooling, thread pooling, etc. be handled automatically. - The low level API should allow the programmer to handle every aspect of the code independently, including handing off requests to different threads, etc. - The ability to write protocol handlers in Tcl, and move them to C if needed for performance. Constraints: - Must be decoupled from the HTTP driver, so that that driver can be changed or replaced at any time without impacting any added drivers Notes: - I'd like to see the Handler API be sufficient at the low level to allow an HTTP handler to be written that is every bit as efficient and speedy as what is being worked on for 4.1 - I'd like to see the high level API be easy enough to use that one could register an "echo server" handler and write the code for it in a dozen lines or so - I'd like the progression from high level to low level be gradual, so the programmer can address only those issues that pertain to his specific protocol Rob Seeger -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 22:25, Dossy Shiobara wrote: > No, we should NOT apply EITHER of these two multi-protocol patches, > > BECAUSE ... the entire driver model is being reworked for 4.1. Can > the same functionality be implemented against 4.1 without getting in > the way of the rest of the code? Nonsense. If any new functionality is applied to the core, it becomes the core. The "getting in the way of the rest of the code" just does not apply. > > BECAUSE ... while both patches make AOLserver do what the original > authors intended, I personally don't think the solution is right. > It may work, but they're both kludges. I personally don't believe > that AOLserver's connection processing model (in either 4.0 or 4.1) > is good for a long-lived TCP connection, but rather geared toward > short-lived TCP connections with one request and one response, such > as with HTTP. Nonsense. This just proves that you haven't seriously examined any of the proposed patches. Before making statements like that, go ahead and read carefully both RFE's and the supplied code. > > > In short, it makes no sense to argue that, "There is no value to these > > patches." We already knows that they DO have value. > > I agree, there is value in the ability to implement high-performance > non-HTTP servers within AOLserver. My personal feelings are that > neither of the two proposed patches do it right. They both piggy-back > on the existing HTTP driver, which is bad because it makes the Nonsense. Vlad's patch implements its own driver entirely. How often must I (or Vlad) say that? > functionality fragile: significant changes or optimizations in the HTTP > driver could totally break non-HTTP servers, leaving anyone relying on > those features out of an upgrade path. Nonsense. Again, if they get in the core, any optimizations in the HTTP driver will/must honour them. But even this is not really true since Vlad's patch entirely bypasses the http processing. > I hope I've given at least one rational reason why I think applying > these patches, in their current form, isn't a great idea. I could not see *any* rational reason from the above. As the matter of fact I do have the evolving impression that you're purposely blocking the effort by giving half-backed, misleading statements based on partial (or even absence of the) evaluation of any of the proposed RFE's. > > I'm glad that people have taken it upon themselves to implement this > functionality for their own projects and have contributed the code back. I strongly doubt that this sentence has any real meaning, besides the political. > It's a great proof-of-concept that it CAN be done. However, I think if > we focused on how to best implement this functionality and code that, > we would be much better off. Think: if Jim Davidson hadn't put nearly > as much thought into the design of AOLserver, would it be as good of an > HTTP server as it is today? Shouldn't we put a similar level of study > into the design of this change rather than just hacking it in "because > it's easy to do this way ..."? I can't speak for Jim, as you're doing. I think it is better to let him (and other AS initiators) speak for themselves... Little bit off topic... In my understanding, AS is good as-is after 4 major releases and 10 years of the development and testing. Still, *significant* changes are done even between minor releases. But see: all are comming from AOL. Where is the community? I can't see any feature on the RFE made itself in the core. I see changes from AOL going into on the regular basis. Why is this so? Is the community just plain lazy? Not inventive? Technically inferior? Is AS a community project or AOL's project? As of today, and this is my experience for the last 4 years, the community served as a great debugging tool for the AOL. Not more. We had already an (non-successful) fork of the code some years ago motivated by the very same reasons as I see are happening now. I think I will open another (flaming) discussion on that, after we settle down on this issue, some time, eventually (if ever). Sometimes I just feel as don Quijote... with a slight difference that windmills are chaging in shape on the regular 2-years basis. (The Enfant Terrible) Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
> a. YES, I like AS to be more powerful multiprotocol > server instead of being a powerful http-server only. > And, I like proposal X better than Y because of Z. I vote for (a). I read both discussions from the sourceforge-links you provided and I see no "jealousness" or "vanity" between the two patch-providers, so I'm optimistic they will provide good information/arguments for a discussion with a final vote. Regards, Bernd. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
> This brings up an interesting point. What kinds of Web sites and > applications do folks typically build with AOLserver? I suspect this type site will become more common in the future, but at present I'd be surprised if most sites being built with AOLserver weren't monolithic, the major exception being credit card settlement... > At AOL we were > discussing using such a feature for sites which include many Web service > calls to other systems, and for other things like server-side ad system > calls, etc. Are you using separate ad server software? or physical server? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
[EMAIL PROTECTED] wrote: On Tue, 2004-08-17 at 17:16, Nathan Folkman wrote: Each addresses different issues. The thread pools are nice in that they give you a way to isolate (by URL I believe?) certain types of activites to particular pools of threads, thus preventing any one paritcular type of activity from utilizing all of the threads. Here's a real world example. Imagine a single Web Server that serves both an application that does a lot of interaction with a database, and also serves up system status pages and other information. Suppose you have the server configured with a max of 10 connection threads. Now imagine a user comes along and decides to do some sort of degenerate query which takes order minutes. The same user, seeing nothing happening, keeps hitting reload. In fact, he's so eager to see some results that he hits the reload button 10 times. You've now got all 10 of your connection threads busy working on his long running database query. Any user wanting to see the system status would not be able to get to the page since the server is now thread maxed. Now imagine you've set things up with thread pools. The database tool gets a max of 5, and the system stats page gets a max of 5. Now our over anxious database application user can only tie up a maximum of 5 connection threads working on his database query, leaving up to 5 connection threads for users wishing to see the system stats page. How do you determine how many threads to add to each pool? Good question, and not one I have a good answer for. Guess the best suggestion I can offer you is make changes based on your application performance. I'd suggest looking at the stats provided by something like "ns_info threads" to see where potential bottle necks are, and to help guide you in determining how many threads you need to partition into seprate thread pools. Let's say you determine the webserver can process 10 requests concurrently. You could assign 5 threads to each pool. In this situation, one pool may be idle and the server may have spare capacity, and yet still reject requests get assigned to the other pool. Alternatively, you could have one pool of 10 threads and classify connections as they are received. You might then specify that both users and admins can max out at 8 conn threads each. Obviously they can't both have 8 conn threads, so what you're effectively specifying here is that the *other* class of connection gets a minimum of 2 conn threads. The effect is more pronounced the more resource pools you have. Interesting. Almost sounds like dynamic thread pool allocation based on system usage? Assuming you could configure the thread pools on the fly, I'm sure you could come up with some clever algorithm which used past thread usage data + the current state to figure out how many threads to allocate to each defined thread pool. Not really sure that would buy you anything though. Suppose the easiest thing to do is monitor and adjust accordingly. It's been my experience that most Web applications experience pretty consist usage patterns. Knowledge of how your applciation functions and how users are using your application should allow you to make a pretty good guess at the optimum configuration. Also, nothing to do with the SEDA or thread pool approach but looking at the current implementation, it's a little inflexible in that it assigns requests to pools based on URL. This allows you to effectively partition your order checkout, for example. It doesn't let you handle quality of service situations like favouring logged in users over anonymous users, or favouring fat-cat CEOs over common proles... :-) A flexible callback interface would be nice to have here. Another good suggestion. You should submit it as a formal RFE with examples of how you'd want to use such an interface. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
[EMAIL PROTECTED] wrote: The SEDA article discusses an approach where you mix a single-threaded event driven piece, in our case it could be the driver thread, with multi-threaded workers, the connection threads. The idea is to avoid having the connection threads, which are relatively precious resources, from sitting around idle waiting on things like I/O. ... The main point of this approach is to move the I/O events up into the single-threaded driver thread, and to avoid waiting on events in your connection threads. That's roughly the idea at least. ;-) Hopefully that makes sense? Yes, it makes sense, though I'm not sure how much sense it makes in the context of the typical kind of website we build with AOLserver. This brings up an interesting point. What kinds of Web sites and applications do folks typically build with AOLserver? At AOL we were discussing using such a feature for sites which include many Web service calls to other systems, and for other things like server-side ad system calls, etc. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
> The SEDA article discusses an approach where you mix a single-threaded > event driven piece, in our case it could be the driver thread, with > multi-threaded workers, the connection threads. The idea is to avoid > having the connection threads, which are relatively precious resources, > from sitting around idle waiting on things like I/O. ... > The main point of this approach is to move the I/O events up into the > single-threaded driver thread, and to avoid waiting on events in your > connection threads. That's roughly the idea at least. ;-) > > Hopefully that makes sense? Yes, it makes sense, though I'm not sure how much sense it makes in the context of the typical kind of website we build with AOLserver. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 17:16, Nathan Folkman wrote: > Each addresses different issues. The thread pools are nice in that they > give you a way to isolate (by URL I believe?) certain types of activites > to particular pools of threads, thus preventing any one paritcular type > of activity from utilizing all of the threads. Here's a real world example. > > Imagine a single Web Server that serves both an application that does a > lot of interaction with a database, and also serves up system status > pages and other information. Suppose you have the server configured with > a max of 10 connection threads. Now imagine a user comes along and > decides to do some sort of degenerate query which takes order minutes. > The same user, seeing nothing happening, keeps hitting reload. In fact, > he's so eager to see some results that he hits the reload button 10 > times. You've now got all 10 of your connection threads busy working on > his long running database query. Any user wanting to see the system > status would not be able to get to the page since the server is now > thread maxed. > > Now imagine you've set things up with thread pools. The database tool > gets a max of 5, and the system stats page gets a max of 5. Now our over > anxious database application user can only tie up a maximum of 5 > connection threads working on his database query, leaving up to 5 > connection threads for users wishing to see the system stats page. How do you determine how many threads to add to each pool? Let's say you determine the webserver can process 10 requests concurrently. You could assign 5 threads to each pool. In this situation, one pool may be idle and the server may have spare capacity, and yet still reject requests get assigned to the other pool. Alternatively, you could have one pool of 10 threads and classify connections as they are received. You might then specify that both users and admins can max out at 8 conn threads each. Obviously they can't both have 8 conn threads, so what you're effectively specifying here is that the *other* class of connection gets a minimum of 2 conn threads. The effect is more pronounced the more resource pools you have. Also, nothing to do with the SEDA or thread pool approach but looking at the current implementation, it's a little inflexible in that it assigns requests to pools based on URL. This allows you to effectively partition your order checkout, for example. It doesn't let you handle quality of service situations like favouring logged in users over anonymous users, or favouring fat-cat CEOs over common proles... :-) A flexible callback interface would be nice to have here. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
[EMAIL PROTECTED] wrote: On Tue, Aug 17, 2004 at 01:32:23PM -0700, Jim Wilcoxson wrote: The reason thread pools would be interesting to me personally is to be able to control quality of service better. For example, I may want a pool of 5 threads to handle search engine spidering requests, and a pool of 5 threads to handle PHP requests, 5 threads to handle a Previously, someone on the list (ah, it was Nathan Folkman) recommended Matt Welsh's SEDA paper: http://www.mail-archive.com/[EMAIL PROTECTED]/msg07026.html http://www.eecs.harvard.edu/~mdw/papers/mdw-phdthesis.pdf http://www.eecs.harvard.edu/~mdw/proj/seda/ That sort of thing sounds both useful and doable in AOLserver. Can you tell us how it would be better than the thread pool mechanism which was added to AOLserver 4? Each addresses different issues. The thread pools are nice in that they give you a way to isolate (by URL I believe?) certain types of activites to particular pools of threads, thus preventing any one paritcular type of activity from utilizing all of the threads. Here's a real world example. Imagine a single Web Server that serves both an application that does a lot of interaction with a database, and also serves up system status pages and other information. Suppose you have the server configured with a max of 10 connection threads. Now imagine a user comes along and decides to do some sort of degenerate query which takes order minutes. The same user, seeing nothing happening, keeps hitting reload. In fact, he's so eager to see some results that he hits the reload button 10 times. You've now got all 10 of your connection threads busy working on his long running database query. Any user wanting to see the system status would not be able to get to the page since the server is now thread maxed. Now imagine you've set things up with thread pools. The database tool gets a max of 5, and the system stats page gets a max of 5. Now our over anxious database application user can only tie up a maximum of 5 connection threads working on his database query, leaving up to 5 connection threads for users wishing to see the system stats page. The SEDA article discusses an approach where you mix a single-threaded event driven piece, in our case it could be the driver thread, with multi-threaded workers, the connection threads. The idea is to avoid having the connection threads, which are relatively precious resources, from sitting around idle waiting on things like I/O. Here's an example that illustrates this. Imagine you've got an application that pulls in data from many sources via simple Web service calls. The driver thread is busy spinning, accepting requests. A request comes to the Web server. A callback is fired off based on the request URL (or something else in the request). The callback fires off a Web service call. The driver thread spins. Perhaps the driver thread accepts a few more requests. The response from the Web service call is received. The driver thread then hands off the initial request plus the data returned from the Web service call to a connection thread where the rest of your application code executes. The main point of this approach is to move the I/O events up into the single-threaded driver thread, and to avoid waiting on events in your connection threads. That's roughly the idea at least. ;-) Hopefully that makes sense? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 17:01, Dossy Shiobara wrote: > On 2004.08.17, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > > So far, the only people who've actually claimed they've read the > > patches are the two authors and Zoran. I'm guessing Dossy has read > > them too but I'm not sure. Did you, Don? > > Yes, I've read both patches. I've even discussed Stephen's patch with > Stephen off-list. The last time we spoke you hadn't read the patch so your comments weren't useful. You still seems to have some misunderstandings about how AOLserver processes requests and how my patch may or may not interact. Ask if you have any questions, that's what we're here to discuss. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > On Tue, Aug 17, 2004 at 01:32:23PM -0700, Jim Wilcoxson wrote: > > > The reason thread pools would be interesting to me personally is to be > > able to control quality of service better. For example, I may want a > > pool of 5 threads to handle search engine spidering requests, and a > > pool of 5 threads to handle PHP requests, 5 threads to handle a > > Previously, someone on the list (ah, it was Nathan Folkman) > recommended Matt Welsh's SEDA paper: > > http://www.mail-archive.com/[EMAIL PROTECTED]/msg07026.html > http://www.eecs.harvard.edu/~mdw/papers/mdw-phdthesis.pdf > http://www.eecs.harvard.edu/~mdw/proj/seda/ > > That sort of thing sounds both useful and doable in AOLserver. Hint: Look at the new driver model in AOLserver 4.1. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > So far, the only people who've actually claimed they've read the > patches are the two authors and Zoran. I'm guessing Dossy has read > them too but I'm not sure. Did you, Don? Yes, I've read both patches. I've even discussed Stephen's patch with Stephen off-list. > You know what? I have a REAL hard time believing that this 147 line > patch could possibly be the complexity and maintenance nightmare that > you're implying, [...] Especially true in C, LOC and complexity are not related. Imagine this one line of code: Ns_FrobSomething(((Ns_Frob *) frobPtr)->thing); Pretend this actually did something. Is it complex? Do you understand what it might actually do? Before you start saying "but nobody writes code like that" ... look inside the AOLserver source. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
> On Tue, Aug 17, 2004 at 01:32:23PM -0700, Jim Wilcoxson wrote: > >> The reason thread pools would be interesting to me personally is to be >> able to control quality of service better. For example, I may want a >> pool of 5 threads to handle search engine spidering requests, and a >> pool of 5 threads to handle PHP requests, 5 threads to handle a > > Previously, someone on the list (ah, it was Nathan Folkman) > recommended Matt Welsh's SEDA paper: > > http://www.mail-archive.com/[EMAIL PROTECTED]/msg07026.html > http://www.eecs.harvard.edu/~mdw/papers/mdw-phdthesis.pdf > http://www.eecs.harvard.edu/~mdw/proj/seda/ > > That sort of thing sounds both useful and doable in AOLserver. Can you tell us how it would be better than the thread pool mechanism which was added to AOLserver 4? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
> I propose this: No one is allowed to claim that any patch either is or > is not a big source of complexity and mainenance headaches unless > they've at least actually read the code in the patch. > > So far, the only people who've actually claimed they've read the > patches are the two authors and Zoran. I'm guessing Dossy has read > them too but I'm not sure. Did you, Don? I haven't, nor am I claiming it will be a "big source". However, it is undeniable that added features mean added complexity and increased maintenance. Big or not. The discussion should center on whether or not the increase is worth it. I happen to think it's not because I personally have no desire to use AOLserver as anything other than a webserver. > I can't seem to checkout the AOLserver head from SourceForge right > now, so I haven't applied it against 4.1 in order to read what it > really does, but according to wc -l, Vlad's patch is a grand total of > 147 lines. Maybe that's why it's a kludge rather than a proper solution, then... > > You know what? I have a REAL hard time believing that this 147 line > patch could possibly be the complexity and maintenance nightmare Please post the specific sentence in which I said it would be a "nightmare". > Arguments about avoiding kludges and the like are worth hearing, but > please, let's not go overboard on the, "Any change at all is > potentially destabilising, evil evil evil!" Again, please post the specific sentence in which I used the word "evil". -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 16:24, Andrew Piskorski wrote: > I can't seem to checkout the AOLserver head from SourceForge right > now, so I haven't applied it against 4.1 in order to read what it > really does, but according to wc -l, Vlad's patch is a grand total of > 147 lines. Anon cvs has been down since Friday so I haven't been able to rediff against the latest changes. Nothing major has changed though, so probably only some minor patch jiggling required. I'll get to it when I can. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, Aug 17, 2004 at 01:32:23PM -0700, Jim Wilcoxson wrote: > The reason thread pools would be interesting to me personally is to be > able to control quality of service better. For example, I may want a > pool of 5 threads to handle search engine spidering requests, and a > pool of 5 threads to handle PHP requests, 5 threads to handle a Previously, someone on the list (ah, it was Nathan Folkman) recommended Matt Welsh's SEDA paper: http://www.mail-archive.com/[EMAIL PROTECTED]/msg07026.html http://www.eecs.harvard.edu/~mdw/papers/mdw-phdthesis.pdf http://www.eecs.harvard.edu/~mdw/proj/seda/ That sort of thing sounds both useful and doable in AOLserver. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, Aug 17, 2004 at 02:42:58PM -0700, [EMAIL PROTECTED] wrote: > > With every version we have bunch of things not working, openssl > > comes to mind, modules and drivers API changed. So this argument > > do not hold water. > > Well, no. The less complex the software we're maintaining, the fewer > maintenances headaches we'll have over time. I propose this: No one is allowed to claim that any patch either is or is not a big source of complexity and mainenance headaches unless they've at least actually read the code in the patch. So far, the only people who've actually claimed they've read the patches are the two authors and Zoran. I'm guessing Dossy has read them too but I'm not sure. Did you, Don? I can't seem to checkout the AOLserver head from SourceForge right now, so I haven't applied it against 4.1 in order to read what it really does, but according to wc -l, Vlad's patch is a grand total of 147 lines. You know what? I have a REAL hard time believing that this 147 line patch could possibly be the complexity and maintenance nightmare that you're implying, especially since Vlad claims that it's very very simple. Heck, my little 'ns_addrbyhost -all' patch is 285 lines! Arguments about avoiding kludges and the like are worth hearing, but please, let's not go overboard on the, "Any change at all is potentially destabilising, evil evil evil!" tack. Stephen's patch is larger, at 1573 lines. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
> Of course they are kludges Well, there we have it, an excellent argument for not adapting these particular patches. > With every version we have bunch of things not working, openssl comes to > mind, > modules and drivers API changed. So this argument do not hold water. Well, no. The less complex the software we're maintaining, the fewer maintenances headaches we'll have over time. That's a given. The argument is whether or not the feature's worth that extra cost, and if the feature IS worth that extra cost, whether the kludge that's proposed is the proper way to implement it. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
> The reason thread pools would be interesting to me personally is to be
> able to control quality of service better. For example, I may want a
> pool of 5 threads to handle search engine spidering requests, and a
> pool of 5 threads to handle PHP requests, 5 threads to handle a
> particular application that uses a different database, 5 threads to
> handle CGI requests, 10 threads to TCL requests, 5 to handle image
> serving (fastpath), 5 to handle all other requests. I'd want to
> dispatch the requests to the various thread pools myself, using TCL
> code. The protocol and communication method (TCP vs UDP) part are
> all the same here - HTTP.
> Perhaps this can all be done today with AS 4 using multiple IP
> addresses - I dunno.
>From sample-config.tcl ... you can map by URL, which means that by proper
partition of your URL-space you can do all you discuss above but shove
requests from spiders into their own thread pool.
#
# Example: Multiple connection thread pools.
#
# To enable:
#
# 1. Define one or more thread pools.
# 2. Configure pools as with the default server pool.
# 3. Map method/URL combinations to the pools
#
# All unmapped method/URL's will go to the default server pool.
#
#ns_section ns/server/server1/pools
#ns_section slow "Slow requests here."
#ns_section fast "Fast requests here."
#
#ns_section ns/server/server1/pool/slow
#ns_param map {POST /slowupload.adp}
#ns_param maxconnections 100 ;# Max connections to put on queue
#ns_param maxdropped 0 ;# Shut down if dropping too many conns
#ns_param maxthreads 20;# Tune this to scale your server
#ns_param minthreads 0 ;# Tune this to scale your server
#ns_param threadtimeout 120 ;# Idle threads die at this rate
#
#ns_section ns/server/server1/pool/fast
#ns_param map {GET /faststuff.adp}
#ns_param maxthreads 10
#
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
> Time and energy have been already spent by implementing/testing those > patches Arguments of this sort apply to EVERY patch and have nothing to do with whether or not it's a good idea to apply them to the code. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Just because things have gone wrong in the past, does not mean we should not work to avoid it in the future. Rob Seeger Vlad Seryakov wrote on 8/17/2004, 4:59 PM: > As for breaking if driver changes, haven't it happened with versions > 3.1, 3.5, 4.0, 4.1. > With every version we have bunch of things not working, openssl comes to > mind, > modules and drivers API changed. So this argument do not hold water. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Of course they are kludges, AOLServer and 4.x versions in particular, designed to be HTTP only. Current AOLServer driver model considers HTTP protocol only and is being optimized to support HTTP or HTTP-like protocols only. I doubt AOL will ever change the whole driver model again to un-HTTP it, i.e. making driver architecture multi-protocol but less HTTP-efficient. That's why, at least my patch, does not change HTTP driver at all, it just makes a little short-cut to stop HTTP processing and passing socket to other protocol handler. It will work with future versions, it works with 4.1 as well. It is similar to existing TCl callback approach, but uses connection pooling, without pre-filters, they are HTTP specific. As for breaking if driver changes, haven't it happened with versions 3.1, 3.5, 4.0, 4.1. With every version we have bunch of things not working, openssl comes to mind, modules and drivers API changed. So this argument do not hold water. Dossy Shiobara wrote: On 2004.08.17, Andrew Piskorski <[EMAIL PROTECTED]> wrote: Therefore, fairly arguing that, "No, we should NOT apply EITHER of these two multi-protocol patches, BECAUSE..." requires that the "BECAUSE" part suggest real, actual NEGATIVES to these patches, and at least the possibility that these negatives may outway the positives. No, we should NOT apply EITHER of these two multi-protocol patches, BECAUSE ... the entire driver model is being reworked for 4.1. Can the same functionality be implemented against 4.1 without getting in the way of the rest of the code? BECAUSE ... while both patches make AOLserver do what the original authors intended, I personally don't think the solution is right. It may work, but they're both kludges. I personally don't believe that AOLserver's connection processing model (in either 4.0 or 4.1) is good for a long-lived TCP connection, but rather geared toward short-lived TCP connections with one request and one response, such as with HTTP. In short, it makes no sense to argue that, "There is no value to these patches." We already knows that they DO have value. I agree, there is value in the ability to implement high-performance non-HTTP servers within AOLserver. My personal feelings are that neither of the two proposed patches do it right. They both piggy-back on the existing HTTP driver, which is bad because it makes the functionality fragile: significant changes or optimizations in the HTTP driver could totally break non-HTTP servers, leaving anyone relying on those features out of an upgrade path. I don't want to recommend people build non-HTTP servers on AOLserver if they won't be properly supported and feature-complete in future versions. I'd rather say "it's unsupported for NOW" than to say "you're supported, but the feature might vaporize in the future if it's inconvenient to support." We already have people who refuse to upgrade from AS 2.x and 3.x and I'm working hard to correct that. I'm pushing hard to get features back into 4.1 to enable folks to upgrade to it from older AOLserver versions. I don't want to see a patch go in now that, in 6 months to a year, cause people to say "I can't upgrade to the latest release because you broke non-HTTP server capability and I rely on it in a production environment, and ..., and ..., and ..." -- can you sympathize with me? What, if any, is the risk of applying these patches? Since they are each in production use on at least one site, and have aleady been independently reviewed by Zoran, my initial assumption is that the risk is pretty low. So far I've seen nothing to suggest otherwise. I hope I've given at least one rational reason why I think applying these patches, in their current form, isn't a great idea. I might be alone here, but that's okay. To summarize: I think the feature being requested is useful and important, but I don't think either of the two patches submitted implement the functionality in such a way that we won't regret it in a future release. I'm glad that people have taken it upon themselves to implement this functionality for their own projects and have contributed the code back. It's a great proof-of-concept that it CAN be done. However, I think if we focused on how to best implement this functionality and code that, we would be much better off. Think: if Jim Davidson hadn't put nearly as much thought into the design of AOLserver, would it be as good of an HTTP server as it is today? Shouldn't we put a similar level of study into the design of this change rather than just hacking it in "because it's easy to do this way ..."? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <
Re: [AOLSERVER] Support for non-HTTP protocols
Ah, but therein lies what I think Dossy takes issue with. No, they don't change the http processing. But... if the http processing changes, the patches can break. I think the general idea Dossy is getting at is to think about a way to implement the functionality such that it is not tied in to the http processing/handler. (Well, that and well thought out, extendable, powerful, yadda yadda yadda ;) Rob Seeger Vlad Seryakov wrote on 8/17/2004, 4:45 PM: > Nobody needs to do anything, AOLServer > still remains http server, those patches do not even change HTTP > processing a bit, they only add additional functionality > which co-exists with existing infrastructure making AOLserver more > versatile environment. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Time and energy have been already spent by implementing/testing those patches, why are we circling around that somebody need to spend efforts to make AOLServer worse HTTP server and better non-HTTP server. Nobody needs to do anything, AOLServer still remains http server, those patches do not even change HTTP processing a bit, they only add additional functionality which co-exists with existing infrastructure making AOLserver more versatile environment. Regarding sendmail list: We use AOLserver with SMTP in front of sendmail and postfix, as MX server, and same aolserver provides us with nice Web interface which simplifies administration to just one server and even that administration goes through Web interface. It is a lot of man-hours for sendmail administrators to achive same level functionality with sendmail/apache-or-another-web-server. I do not vote but regardless of the decision i will maintain my multi-protocol version of AOLServer4 (i have many production servers). Dossy Shiobara wrote: On 2004.08.17, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: IMO, this is all BS. I don't mean any disrespect, but if I wanted a mail server, I'd install a mail server. I don't go on the sendmail list screaming for it to support http :) Let me rephrase what I think Donald was trying to say, but without the edgy-ness: If we could focus our energies on making AOLserver a better HTTP server, that would be really cool. It's the "needs of the many" vs "needs of the few" issue -- yes, making AOLserver a better non-HTTP server serves the needs of the few, but if we're going to spend time and energy, why can't we do it to make AOLserver a better HTTP server than it already is? That would benefit far more people who already use AOLserver ... -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- Vlad Seryakov 703 488-2173 office [EMAIL PROTECTED] http://www.crystalballinc.com/vlad/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 11:51 am, Dossy Shiobara wrote: Janine has offered to help, but cannot provide access to her production environments where the problem is occurring. Actually I can, but what I can't do is put AOLserver 4 + nsopenssl beta 21 back into production. Right now I am running 3.3+ad13 all around, with one of the staging sites on 3.5.11 to see if it will work ok. I don't think being able to log into this box now is going to help you much with the nsopenssl problem. I'll post an update later on the random hang problem I've been having with my current setup, the one that prompted my attempt to upgrade in the first place. janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
> On Tue, Aug 17, 2004 at 08:55:26AM -0400, Dossy Shiobara wrote: > For example, arguments along the lines of: > - The patches will make the code more complicated, uglier, bug prone, > or harder to maintain. Three of the four points above are a given. 1. The code will be more complicated. 2. More code and new features increase the probability of bugs. 3. Yes, it will be harder to maintain, because the new features will need to be tested for each release. I have no idea if the code's ugly or not. > In short, it makes no sense to argue that, "There is no value to these > patches." We already knows that they DO have value. Every proposed patch has value by your definition, since you argued the proof of value is that someone wrote the code that goes into the patch. This doesn't mean that every patch implements a good idea, though. > Flexibility, power, and options are good. This is commonly known as the MicroSoft definition of software quality. It's all about features, not about the appropriateness of implementing those features in a particular piece of software. > Giving people the option to > code non-HTTP servers in C using AOLserver is GOOD. Why? All I want is a simple, reliable, efficient web server that provides me with a nice DB interface and a built-in interpretive language. Is also good to give people the option to do their word processing in AOLserver? If not, why not, and where's the intrinsic difference in goodness between one case and the other? > If people see > real problems in these patches to do that, yes please, speak up, we > need to hear about it! But the AOLserver project also has a LONG > history of opposing useful, important patches (even bug fixes!) from > outside developers, often for no apparent reason at all, and that sort > of "not invented here" syndrome gets awfully tiresome. Where's the NIH syndrome here? This represents a major change in philosophy, that AOLserver is no longer a webserver but something entirely different. Has nothing to do with who invented the patch. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
>From my point of view, focusing on protocols is not necessarily the only reason to want thread pools. I'm not all that familiar with AS 4 yet, so maybe I'm being dense here... keep that in mind. :) The reason thread pools would be interesting to me personally is to be able to control quality of service better. For example, I may want a pool of 5 threads to handle search engine spidering requests, and a pool of 5 threads to handle PHP requests, 5 threads to handle a particular application that uses a different database, 5 threads to handle CGI requests, 10 threads to TCL requests, 5 to handle image serving (fastpath), 5 to handle all other requests. I'd want to dispatch the requests to the various thread pools myself, using TCL code. The protocol and communication method (TCP vs UDP) part are all the same here - HTTP. Perhaps this can all be done today with AS 4 using multiple IP addresses - I dunno. However, sometimes it would be useful to have the pool functionality with one IP address. If a search engine decides to aggressively index your site, they have to do it on the main IP address. Being able to give them a different class of service, ie, a different thread pool, would be useful. We've already done a similar thing, by setting up multiple AS instances. But it would have been easier in many cases to set it up with thread pools. Much less code on our end to change. A way to accomplish the same thing today is by putting an intelligent switch/cache/proxy device in front of multiple servers, and having it route certain kinds of requests to certain servers. The problem from my point of view is: - the switch is another device to manage - it's an extra $5000+ - it's often a proprietary solution - if you look at client IP addresses, you have IP address issues to deal with, or have to set up some fancy routing, or have to modify your code to get the true peer address from a made-up header - you have to learn how to use the device's routing rules instead of making rules yourself, with TCL - hit logging will probably work differently; another difference to handle Another advantage to using thread pools in this application is that it may help a site be less vulnerable to DOS attacks. If a site is running 3rd party bulletin board software for example, and someone decides to run an attack against it, it may still kill that the board service/thread pool, but the other services running on the same server instance would not be unaffected. Without this partitioning, the attack might use all of the threads and kill the site. Maybe the thread pools could also have priorities too. So I could make dispatching decisions based on the current request load, like, "If there are more than 5 threads active in the main pool, hold off executing any new requests in the bulletin board pool, ie, queue the request, but don't let it execute. In an ideal world, there would always be enough resource to do everything, but in the real world, there are sometimes momentary resource shortages that cause things to get slow. If something is going to be slow, I'd rather it be a non-essential service like a bulletin board rather than the threads generating our main pages. Is this something critical I can't live without? No. But it seems useful. Jim > I definitely vote for a). I think AS has a very > powerful framework that can be used outside of http. > > --- Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: > > > On Tuesday 17 August 2004 10:32, Zoran Vasiljevic > > wrote: > > > Hi! > > > > > > I would like to start a discussion which will > > eventually lead > > > to one of the two proposals listed below (one from > > Vlad Seryakov > > > and the other from Stephen Deasey) included in the > > core-server > > > distribution. > > > > Eh... wrong turn again... > > By "start a discussion", I really expected the > > people to > > respond with: > > > > a. YES, I like AS to be more powerful > > multiprotocol > > server instead of being a powerful http-server > > only. > > And, I like proposal X better than Y because of > > Z. > > > > b. NO, I do not need AS doing anything else than > > http > > and am perfectly happy with status-quo now. > > Just > > forget the whole thing. > > > > I apologize for not being more precise. > > > > If a. turns out to be everbody's darling, then there > > are > > two excellent proposals for making this work. If > > somebody > > has a better solution, lets see this as RFE. Then, > > get a > > couple of (knowledgeable) people vote in democratic > > way > > about what's the better solution. I think there is a > > AOLSERVER-DEVEL (or such) list where deeper > > technical > > issues can be discussed in a smaller audience. > > > > If b. well, than it is clear. We just leave it > > as-is. > > > > Can we agree on this? > > > > Zoran > > > > > > -- > > AOLserver - http://www.aolserver.com/ > > > > To Remove yourself from this list, simply send an > > email to <[EMAIL PROTECTED]> with the
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > > Therefore, fairly arguing that, "No, we should NOT apply EITHER of > these two multi-protocol patches, BECAUSE..." requires that the > "BECAUSE" part suggest real, actual NEGATIVES to these patches, and at > least the possibility that these negatives may outway the positives. No, we should NOT apply EITHER of these two multi-protocol patches, BECAUSE ... the entire driver model is being reworked for 4.1. Can the same functionality be implemented against 4.1 without getting in the way of the rest of the code? BECAUSE ... while both patches make AOLserver do what the original authors intended, I personally don't think the solution is right. It may work, but they're both kludges. I personally don't believe that AOLserver's connection processing model (in either 4.0 or 4.1) is good for a long-lived TCP connection, but rather geared toward short-lived TCP connections with one request and one response, such as with HTTP. > In short, it makes no sense to argue that, "There is no value to these > patches." We already knows that they DO have value. I agree, there is value in the ability to implement high-performance non-HTTP servers within AOLserver. My personal feelings are that neither of the two proposed patches do it right. They both piggy-back on the existing HTTP driver, which is bad because it makes the functionality fragile: significant changes or optimizations in the HTTP driver could totally break non-HTTP servers, leaving anyone relying on those features out of an upgrade path. I don't want to recommend people build non-HTTP servers on AOLserver if they won't be properly supported and feature-complete in future versions. I'd rather say "it's unsupported for NOW" than to say "you're supported, but the feature might vaporize in the future if it's inconvenient to support." We already have people who refuse to upgrade from AS 2.x and 3.x and I'm working hard to correct that. I'm pushing hard to get features back into 4.1 to enable folks to upgrade to it from older AOLserver versions. I don't want to see a patch go in now that, in 6 months to a year, cause people to say "I can't upgrade to the latest release because you broke non-HTTP server capability and I rely on it in a production environment, and ..., and ..., and ..." -- can you sympathize with me? > What, if any, is the risk of applying these patches? Since they are > each in production use on at least one site, and have aleady been > independently reviewed by Zoran, my initial assumption is that the > risk is pretty low. So far I've seen nothing to suggest otherwise. I hope I've given at least one rational reason why I think applying these patches, in their current form, isn't a great idea. I might be alone here, but that's okay. To summarize: I think the feature being requested is useful and important, but I don't think either of the two patches submitted implement the functionality in such a way that we won't regret it in a future release. I'm glad that people have taken it upon themselves to implement this functionality for their own projects and have contributed the code back. It's a great proof-of-concept that it CAN be done. However, I think if we focused on how to best implement this functionality and code that, we would be much better off. Think: if Jim Davidson hadn't put nearly as much thought into the design of AOLserver, would it be as good of an HTTP server as it is today? Shouldn't we put a similar level of study into the design of this change rather than just hacking it in "because it's easy to do this way ..."? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 11:51 am, Dossy Shiobara wrote: > Janine has offered to help, but cannot provide access to her production > environments where the problem is occurring. Sloan's going to try to reproduce it in the next week or two, at which time I plan to dig around in gdb ... -- Don Baccus Portland, OR http://donb.furfly.net, http://birdnotes.net, http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >IMO, this is all BS. I don't mean any disrespect, but if I wanted > a mail server, I'd install a mail server. I don't go on the sendmail > list screaming for it to support http :) Let me rephrase what I think Donald was trying to say, but without the edgy-ness: If we could focus our energies on making AOLserver a better HTTP server, that would be really cool. It's the "needs of the many" vs "needs of the few" issue -- yes, making AOLserver a better non-HTTP server serves the needs of the few, but if we're going to spend time and energy, why can't we do it to make AOLserver a better HTTP server than it already is? That would benefit far more people who already use AOLserver ... -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, Aug 17, 2004 at 07:17:52PM +0200, Zoran Vasiljevic wrote: > a. YES, I like AS to be more powerful multiprotocol > server instead of being a powerful http-server only. > And, I like proposal X better than Y because of Z. I'd like to hear more discussion first, but so far my vote is "a." This is a potentially very powerful and useful feature. I don't have any specific need for it right now, but I can easily imagine it being exactly the right tool for some future project. >From Zoran's description, Stephen Deasey's patch sounds the better of the two, but I haven't looked at any of this code myself at all, so what that really boils down to is, "I defer to Zoran's judgement." -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, Aug 17, 2004 at 08:55:26AM -0400, Dossy Shiobara wrote: > I see two "classes" of people using AOLserver for implementing non-HTTP > servers: > > 1) Low to medium performance requirements, short development timeline, >need for rapid development, programmers with low to average technical >expertise. > > 2) High performance requirements, highly skilled programmers, moderate >development timeline, scalability a major concern. Dossy, this is all very interesting, but it also seems to have rather little to do with Zoran's original topic. Let's be blunt here: We have two proposed patches on the table for enabling AOLserver to be a multi-protocol rather than HTTP-only server. The question at hand is should either of these patches be applied, and if so, which one. So, Dossy, are you SERIOUSLY arguing that the correct answer is, "No, don't apply EITHER patch, because you can always roll your own from scratch in pure Tcl."? All your discussion so far can be easily taken to imply that, but I've honestly no idea whether that's your position or not. Please clarify. Finally, to head off the next and obvious question: "No, don't apply EITHER patch, because you can always roll your own from scratch in plain Tcl", is, IMNSHO, an totally bogus, unnacceptable argument. Yes, perhaps you COULD implement XYZ in Tcl only without any further integration into the existing AOLserver C facilities, and MAYBE it might even be almost as fast as doing it in C - it could even be true. But it's also almost COMPLETELY irrelevent to the question of, "Should we apply one of these two patches." Here's why: We already know that SOME hard-core AOLserver users find extended multi-protocol support in AOLserver, implemented in C, to be very useful. We know this because they went to the trouble to implement it themselves, are using it on their production sites, have discussed how it's important to them many times on this list, and have posted patches for public review and adoption. There is *NO* debate about, "Can this feature be significantly useful to at least some people?" We already know beyond any reasonable doubt that the answer to that is, "Yes!". Therefore, fairly arguing that, "No, we should NOT apply EITHER of these two multi-protocol patches, BECAUSE..." requires that the "BECAUSE" part suggest real, actual NEGATIVES to these patches, and at least the possibility that these negatives may outway the positives. For example, arguments along the lines of: - The patches will make the code more complicated, uglier, bug prone, or harder to maintain. - We don't and won't have to do it, and we won't anytime soon, either. In short, it makes no sense to argue that, "There is no value to these patches." We already knows that they DO have value. What we may not know yet is, how much value, what might the costs and risks be, is the value worth those risks, and finally, what should we do about it. What, if any, is the risk of applying these patches? Since they are each in production use on at least one site, and have aleady been independently reviewed by Zoran, my initial assumption is that the risk is pretty low. So far I've seen nothing to suggest otherwise. Flexibility, power, and options are good. Giving people the option to code non-HTTP servers in C using AOLserver is GOOD. If people see real problems in these patches to do that, yes please, speak up, we need to hear about it! But the AOLserver project also has a LONG history of opposing useful, important patches (even bug fixes!) from outside developers, often for no apparent reason at all, and that sort of "not invented here" syndrome gets awfully tiresome. Please note, I'm NOT claiming that's what's going on here, right now. I don't really think it is actually, at least not yet. I just want to do my little bit to help keep us from heading off down that unpleasant road once again... -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
I definitely vote for a). I think AS has a very powerful framework that can be used outside of http. --- Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: > On Tuesday 17 August 2004 10:32, Zoran Vasiljevic > wrote: > > Hi! > > > > I would like to start a discussion which will > eventually lead > > to one of the two proposals listed below (one from > Vlad Seryakov > > and the other from Stephen Deasey) included in the > core-server > > distribution. > > Eh... wrong turn again... > By "start a discussion", I really expected the > people to > respond with: > > a. YES, I like AS to be more powerful > multiprotocol > server instead of being a powerful http-server > only. > And, I like proposal X better than Y because of > Z. > > b. NO, I do not need AS doing anything else than > http > and am perfectly happy with status-quo now. > Just > forget the whole thing. > > I apologize for not being more precise. > > If a. turns out to be everbody's darling, then there > are > two excellent proposals for making this work. If > somebody > has a better solution, lets see this as RFE. Then, > get a > couple of (knowledgeable) people vote in democratic > way > about what's the better solution. I think there is a > AOLSERVER-DEVEL (or such) list where deeper > technical > issues can be discussed in a smaller audience. > > If b. well, than it is clear. We just leave it > as-is. > > Can we agree on this? > > Zoran > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an > email to <[EMAIL PROTECTED]> with the > body of "SIGNOFF AOLSERVER" in the email message. > You can leave the Subject: field of your email > blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 20:55, Don Baccus wrote: > Fine. But there's no reason to push forward on this immediately. Resources > are finite, triage and prioritization are important Of course. There is no need to push this. It is, if at all, going to be the part of the development branch for 4.1 or some future release. BTW, the job is already done by Vlad/Stephen. There is no additional work to be done by anybody, except doing the merge in the CVS. People not interested in, or having no need for it, will see no difference in speed or stability of the server. It is a clear win-win situation. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Don Baccus <[EMAIL PROTECTED]> wrote: > And, yes, I realize OpenSSL isn't part of the AOLserver core proper, > but it's importance is such that it might as well be. I've been trying, with little luck, to reproduce the problems people have been seeing with nsopenssl. Janine has offered to help, but cannot provide access to her production environments where the problem is occurring. If anyone can even posit hypothesis as to what's causing the error, I'd be more than happy to investigate. Within AOL, we use nsopenssl 3 beta and I haven't heard nary a peep from the Ops folks that they're seeing the same problems that are being observed in the OSS community. I'm beginning to suspect a configuration-related issue ... -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
[EMAIL PROTECTED] wrote: On Tuesday 17 August 2004 10:59 am, Zoran Vasiljevic wrote: Why is this on the table? Because there are people looking for this kind of functionality and because two of them invested great deal of time to get it working for themselves. And, they posted a RFE and are willing to share their work. Fine. But there's no reason to push forward on this immediately. Resources are finite, triage and prioritization are important, and we have definite breakage in the current supposedly stable release - OpenSSL is crucial to any commercial usage for e-commerce or any environment where sensitive information is passed to or taken from the user's browser. And, yes, I realize OpenSSL isn't part of the AOLserver core proper, but it's importance is such that it might as well be. Have you looked at nsnss? ;-) Some highlights from the Changelog: $Header: /cvsroot/aolserver/nsnss/README,v 1.1.1.1 2004/04/12 13:46:25 rcrittenden0569 Exp $ SSLv2, SSLv3, TLSv1 Module -- This module requires NSPR v4.4.1 and NSS v3.9 or higher. It will work with lower versions of NSS and NSPR but some tweaks may be required. Feature Highlights -- * Open Source software (AOLserver Public License or GPL) * Useable for both commercial and non-commercial use * 128-bit strong cryptography world-wide * Support for SSLv2, SSLv3 and TLSv1 protocols * Support for both RSA and Diffie-Hellman ciphers * FIPS-certified SSL implementation * Support for client certificate verification * Clean, reviewable ANSI C source code -- Don Baccus Portland, OR http://donb.furfly.net, http://birdnotes.net, http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 10:59 am, Zoran Vasiljevic wrote: > Why is this on the table? Because there are people looking for > this kind of functionality and because two of them invested > great deal of time to get it working for themselves. And, they > posted a RFE and are willing to share their work. Fine. But there's no reason to push forward on this immediately. Resources are finite, triage and prioritization are important, and we have definite breakage in the current supposedly stable release - OpenSSL is crucial to any commercial usage for e-commerce or any environment where sensitive information is passed to or taken from the user's browser. And, yes, I realize OpenSSL isn't part of the AOLserver core proper, but it's importance is such that it might as well be. -- Don Baccus Portland, OR http://donb.furfly.net, http://birdnotes.net, http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 20:12, Don Baccus wrote: > On Tuesday 17 August 2004 10:17 am, Zoran Vasiljevic wrote: > > > b. NO, I do not need AS doing anything else than http > > and am perfectly happy with status-quo now. Just > > forget the whole thing. > > This is my vote. Let's keep things simple and let's keep things working. Why > is this even on the table when people using OpenSSL can't run AOLserver 4 in > a production environment? > Thanks, Don. Why is this on the table? Because there are people looking for this kind of functionality and because two of them invested great deal of time to get it working for themselves. And, they posted a RFE and are willing to share their work. I can't say much about openssl problems, though. Zoran > -- > Don Baccus > Portland, OR > http://donb.furfly.net, http://birdnotes.net, http://openacs.org > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 10:17 am, Zoran Vasiljevic wrote: > b. NO, I do not need AS doing anything else than http > and am perfectly happy with status-quo now. Just > forget the whole thing. This is my vote. Let's keep things simple and let's keep things working. Why is this even on the table when people using OpenSSL can't run AOLserver 4 in a production environment? -- Don Baccus Portland, OR http://donb.furfly.net, http://birdnotes.net, http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 10:32, Zoran Vasiljevic wrote: > Hi! > > I would like to start a discussion which will eventually lead > to one of the two proposals listed below (one from Vlad Seryakov > and the other from Stephen Deasey) included in the core-server > distribution. Eh... wrong turn again... By "start a discussion", I really expected the people to respond with: a. YES, I like AS to be more powerful multiprotocol server instead of being a powerful http-server only. And, I like proposal X better than Y because of Z. b. NO, I do not need AS doing anything else than http and am perfectly happy with status-quo now. Just forget the whole thing. I apologize for not being more precise. If a. turns out to be everbody's darling, then there are two excellent proposals for making this work. If somebody has a better solution, lets see this as RFE. Then, get a couple of (knowledgeable) people vote in democratic way about what's the better solution. I think there is a AOLSERVER-DEVEL (or such) list where deeper technical issues can be discussed in a smaller audience. If b. well, than it is clear. We just leave it as-is. Can we agree on this? Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Of course, we're both only speculating here. Our production driver is publicily available http://www.crystalballinc.com/vlad/software/maverix/ You can see if implementing it in Tcl would be a better solution but keep in mind that it was designed to be working under heavy load. Sometimes it is processing 200 req/sec on one 2-CPU Dell server. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 10:04, Dossy Shiobara wrote: > On 2004.08.17, Stephen Deasey <[EMAIL PROTECTED]> wrote: > > This isn't true. In AOLserver, once a request has completed the conn is > > returned to the driver thread for keepalive. It's monitored in a > > non-blocking fashion until either it times out or the client makes > > another request. In the meantime, the conn thread which just completed > > is free to service other requests. > > Um, what about protocols that requires the server to maintain state with > the connection? If the thread servicing the connection gets returned to > the pool, doesn't the state go along with it? State is stored in session local storage. This is just like conn and thread local storage, but the data lifetime is from socket connect to socket disconnect. The example server illustrates this. > > This makes it possible to have 1000 concurrent connections serviced by a > > pool of just 10 threads -- at any one time most of the connections will > > be momentarily idle. This benefits memory usage and context switch > > overhead. > > This is only desirable for stateless protocols, like HTTP. It's highly desirable for stateful protocols too. The inactivity timeout for POP3 is 10 minutes, so potentially your thread+interp could be kept tied up for that period of time. My email client (IMAP) checks for new messages every 3 minutes -- it doesn't disconnect in between. The request-response-pause pattern is common to many protocols. > > This is one of the major architectural benefits AOLserver has over > > something like Apache. In the Apache model, processes take turns > > listening for connections. When a connection arrives, the process > > switches roles from a listener to a worker and processes the > > connection. The worker processes the connection to completion, > > including idle wait time due to keepalive. > > True, for stateless protocols like HTTP, the AOLserver model is > superior. For stateful protocols, this may not work so well. > > > AOLserver is ideally suited to the majority of server tasks. About the > > only shortcoming which comes to mind is that conn threads are required > > to do blocking writes. But fixing that would be of benefit to the HTTP > > processing side too. > > So, we want reader threads and separate writer threads now, too? > Exactly how many free-range threads do we want roaming the server > prairie, here? > > > > Of course, we're both only speculating here. > > > > Speak for yourself! > > Show me your measurements. And I'm not talking 36D-28-34, either. I think the burden of proof lies with you. A non-HTTP server implemented with the proposed Ns_ParseProc interface is exactly the same architecturally as stock AOLserver. If you think you can rewrite it all in Tcl and make it faster, I'd be interested to see that. I don't think you can do it. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Stephen Deasey <[EMAIL PROTECTED]> wrote: > This isn't true. In AOLserver, once a request has completed the conn is > returned to the driver thread for keepalive. It's monitored in a > non-blocking fashion until either it times out or the client makes > another request. In the meantime, the conn thread which just completed > is free to service other requests. Um, what about protocols that requires the server to maintain state with the connection? If the thread servicing the connection gets returned to the pool, doesn't the state go along with it? > This makes it possible to have 1000 concurrent connections serviced by a > pool of just 10 threads -- at any one time most of the connections will > be momentarily idle. This benefits memory usage and context switch > overhead. This is only desirable for stateless protocols, like HTTP. > This is one of the major architectural benefits AOLserver has over > something like Apache. In the Apache model, processes take turns > listening for connections. When a connection arrives, the process > switches roles from a listener to a worker and processes the > connection. The worker processes the connection to completion, > including idle wait time due to keepalive. True, for stateless protocols like HTTP, the AOLserver model is superior. For stateful protocols, this may not work so well. > AOLserver is ideally suited to the majority of server tasks. About the > only shortcoming which comes to mind is that conn threads are required > to do blocking writes. But fixing that would be of benefit to the HTTP > processing side too. So, we want reader threads and separate writer threads now, too? Exactly how many free-range threads do we want roaming the server prairie, here? > > Of course, we're both only speculating here. > > Speak for yourself! Show me your measurements. And I'm not talking 36D-28-34, either. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 08:07, Dossy Shiobara wrote: > On 2004.08.17, Stephen Deasey <[EMAIL PROTECTED]> wrote: ... > > Each of your connection threads and all it's resources are committed for > > the life of the connection. It will remain committed until the client > > disconnects. > > This will be true regardless of the implementation (C vs. Tcl) as it's a > requirement of the TiVo beacon protocol -- clients maintain persistent > connections to the server. This isn't true. In AOLserver, once a request has completed the conn is returned to the driver thread for keepalive. It's monitored in a non-blocking fashion until either it times out or the client makes another request. In the meantime, the conn thread which just completed is free to service other requests. This makes it possible to have 1000 concurrent connections serviced by a pool of just 10 threads -- at any one time most of the connections will be momentarily idle. This benefits memory usage and context switch overhead. This is one of the major architectural benefits AOLserver has over something like Apache. In the Apache model, processes take turns listening for connections. When a connection arrives, the process switches roles from a listener to a worker and processes the connection. The worker processes the connection to completion, including idle wait time due to keepalive. > I disagree. SMTP and IMAP are toy servers with regard to complexity, > just as the TiVo server is. They are all very simple protocols. The > challenge is scalability -- the assumption here is that a pure Tcl > solution will not perform well enough to scale compared to the C > solution. It's only an assumption. As you say, parsing is the least difficult part of creating a scalable server. Which is why I added a hook to implement just that, so that you can take advantage of all the AOLserver scalability features. > My hunch is that empirical evidence will show that none of the > "features" of AOLserver will make the C solution perform exceptionally > better than a pure Tcl solution. If you're going to implement in C for > performance, implement something tailored to solving the problem > specifically -- that will be the only way to really get the performance > required. AOLserver is ideally suited to the majority of server tasks. About the only shortcoming which comes to mind is that conn threads are required to do blocking writes. But fixing that would be of benefit to the HTTP processing side too. > Of course, we're both only speculating here. Speak for yourself! -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
It is not thread pools by itself but more reusing existing thread pools. If you have AOLServer thread pools and Tcl thread pools and if you have 100 connections always connected to the server and using SMTP and HTTP (this is my case) both AOLServer pools and Tcl pools would compete for the same resources. I do not have Tcl implementation, it was more than 1 year ago and i did not save old experimental code. Dossy Shiobara wrote: On 2004.08.17, Vlad Seryakov <[EMAIL PROTECTED]> wrote: Only after i started using AOLServer's thread pools and connection manager, performance issues disappear. Perhaps I should share code to implement thread pools in pure Tcl? Do you still have your pure Tcl implementation of your SMTP server? Would you be interested in hooking a thread pool implementation up to it and see how it performs? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- Vlad Seryakov 703 488-2173 office [EMAIL PROTECTED] http://www.crystalballinc.com/vlad/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Robert Seeger <[EMAIL PROTECTED]> wrote: > Rather than implement thread pools in pure Tcl, why not just expose AS' > thread pool implementation at the Tcl level? That way, in the end, > there's only one code base to test and update. > > Keep in mind, I'm not saying its better, I'm really just curious. Yes, that would be the longer-term solution. However, to prove it's worth investing in the longer-term solution, implementing thread pools in pure Tcl would hopefully try to "prove" that having such a thing would solve one of the bottlenecks to otherwise pure Tcl solutions. If, upon implementing thread pools in pure Tcl, we found that performance was still insufficient in a pure Tcl implementation, we'd know that it's not worth exposing AOLserver's C implementation with a Tcl interface. However, if the contrary is found to be true, then yes, we'd work to abstract out AOLserver's thread pooling and expose it with a Tcl API. If we're lucky, we do it in a backwards-compatible fashion as a bonus ... -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 10:32, Zoran Vasiljevic wrote: > I would like to start a discussion which will eventually lead > to one of the two proposals listed below (one from Vlad Seryakov > and the other from Stephen Deasey) included in the core-server > distribution. > I anticipated that this one may evolve in large discussion. Just to narrow down... The proposal is *not* about if XYZ can be done in Tcl and/or how much slower or faster this could be. This is *not* the intention. The intention is to augment, on the architecture-level, the http-only protocol server, as AS server now is, to a protocol server, with the option to plug_your_own minimal implementation and benefit from all other stuff: logging, thread/conn pools, caches, etc. This opens whole lotta application options and allows people to build upon the great functionality already in the server w/o re-inventing the wheel (in Tcl) over and over again. Hm? Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Rather than implement thread pools in pure Tcl, why not just expose AS' thread pool implementation at the Tcl level? That way, in the end, there's only one code base to test and update. Keep in mind, I'm not saying its better, I'm really just curious. Rob Seeger Dossy Shiobara wrote on 8/17/2004, 10:13 AM: > On 2004.08.17, Vlad Seryakov <[EMAIL PROTECTED]> wrote: > > Only after i started using AOLServer's thread pools and > > connection manager, performance issues disappear. > > Perhaps I should share code to implement thread pools in pure Tcl? > Do you still have your pure Tcl implementation of your SMTP server? > Would you be interested in hooking a thread pool implementation up to it > and see how it performs? > > -- Dossy > > -- > Dossy Shiobara mail: [EMAIL PROTECTED] > Panoptic Computer Network web: http://www.panoptic.com/ > "He realized the fastest way to change is to laugh at your own > folly -- then you can let go and quickly move on." (p. 70) > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to > <[EMAIL PROTECTED]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the > Subject: field of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 16:07, Dossy Shiobara wrote: > Of course, we're both only speculating here. Vlad is not speculating: > This is what i 've done with my first version of SMTP driver, i wrote it > in pure Tcl. Performance was terrible, the site was serving > thousands of incoming SMTP connections and it was very slow. Second > version was doing pretty same stuff but in C, i registered TCP callback > and create new thread for each new connection. Not much a difference. > Only after i started using AOLServer's thread pools and > connection manager, performance issues disappear. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Vlad Seryakov <[EMAIL PROTECTED]> wrote: > Only after i started using AOLServer's thread pools and > connection manager, performance issues disappear. Perhaps I should share code to implement thread pools in pure Tcl? Do you still have your pure Tcl implementation of your SMTP server? Would you be interested in hooking a thread pool implementation up to it and see how it performs? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Stephen Deasey <[EMAIL PROTECTED]> wrote: > The performance difference would certainly be noticeable. As long as we're clear that we're BOTH speculating here, then I'm okay with your statement. If you're asserting this to be true, where's the proof? > In your example code a new thread is spawned for each connection. This > can take a non trivial amount of time, especially if a new interp has to > be initialized. I could, inside 50 lines of code, implement my own thread pool in pure Tcl. I didn't need it, so I didn't bother. > Each of your connection threads and all it's resources are committed for > the life of the connection. It will remain committed until the client > disconnects. This will be true regardless of the implementation (C vs. Tcl) as it's a requirement of the TiVo beacon protocol -- clients maintain persistent connections to the server. > There are some functional differences too: no access logging; none of > the niceties of registered procs or filters; no caching of returned > files, etc. etc. Access logging would be easy to implement if the nslog module was extended to expose a Tcl command to generate log entries that nslog would write. Probably not a bad idea for an RFE. Registered procs and filters are irrelevant for my specific case, but again, an exposed Tcl command that allowed one to execute the procs or filters registered for a user-provided URL would make this possible with two lines of Tcl in my code. Caching of returned files -- in my case, the responses are fully dynamic. My point is, I'm implementing a non-HTTP server. Many of the features of AOLserver that make it an excellent choice as an HTTP server are irrelevant if you're not implementing an HTTP server. > None of which matters for a toy like a Tivo server. An SMTP or IMAP > server implemented like this wouldn't be worth the bother. I disagree. SMTP and IMAP are toy servers with regard to complexity, just as the TiVo server is. They are all very simple protocols. The challenge is scalability -- the assumption here is that a pure Tcl solution will not perform well enough to scale compared to the C solution. It's only an assumption. My hunch is that empirical evidence will show that none of the "features" of AOLserver will make the C solution perform exceptionally better than a pure Tcl solution. If you're going to implement in C for performance, implement something tailored to solving the problem specifically -- that will be the only way to really get the performance required. Of course, we're both only speculating here. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
This is what i 've done with my first version of SMTP driver, i wrote it
in pure Tcl. Performance was terrible, the site was serving
thousands of incoming SMTP connections and it was very slow. Second
version was doing pretty same stuff but in C, i registered TCP callback
and create new thread for each new connection. Not much a difference.
Only after i started using AOLServer's thread pools and
connection manager, performance issues disappear. So if new protocol
does not serve thousands of connections, Tcl/TCP callback
solution works just fine.
Dossy Shiobara wrote:
On 2004.08.17, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote:
So, all of you interested in extending AS to handle non-http
protocols, please go, have a look and give some feedback.
Just the other night, I had the need to implement a non-HTTP server in
AOLserver in order to implement the TiVo "beacon" connection for the
Home Media Option (MHO) feature of the Series2 TiVo. I thought about
both Vlad's and Stephen's approaches, and decided I'd rather stay
pure-Tcl instead. Here's what I came up with:
proc tivo.init {} {
# ...some setting of nsv config vars here...
ns_thread begindetached tivo.beaconThread
}
proc tivo.beaconThread {} {
set address [nsv_get tivo address]
set beaconport [nsv_get tivo beaconport]
ns_log notice "TiVo beacon listening on $address:$beaconport"
set master [ns_socklisten $address $beaconport]
while {![nsv_get tivo shutdown]} {
foreach {rfd wfd} [ns_sockaccept $master] break
ns_chan create $rfd $rfd
ns_chan create $wfd $wfd
ns_thread begindetached [list tivo.beacon $rfd $wfd]
}
close $master
}
tivo.beacon {rfd wfd} {
ns_chan get $rfd
ns_chan get $wfd
# ... do actual beacon exchange ...
# connection must remain open, so we just poll until the remote
# side disconnects.
while {![nsv_get tivo shutdown]} {
set sel [ns_sockselect -timeout 2 $rfd {} {}]
if {[llength [lindex $sel 0]]} {
read $rfd 1
if {[fblocked $rfd] || [eof $rfd]} {
ns_log notice "TiVo: Beacon disconnected."
break
}
}
}
close $rfd
close $wfd
}
The gist of this is that:
1. Can't use [ns_socklistencallback] since our connection handler keeps
the connection open, and that would block the thread that executes
the callback handler.
2. At server start-up, we create a thread to just accept connections
which creates new threads to process those connections.
3. We use [ns_chan] to detach/attach Tcl channels to the socket fd's in
order to pass them from the master thread to the slave threads.
So far, this has been working really well for me. No modification to
the core involved, and it's implemented in very few LOC overall. Using
the same pattern, I could easily implement an SMTP, IMAP, etc., server
in pure Tcl. I'm not sure what the real performance difference there
would be compared to the two proposals that require C code changes in
the core: as long as the actual "functionality" is implemented in a Tcl
callback that gets invoked from the C code, the performance difference
is probably negligible.
On a side note: if you have a Series2 TiVo with Home Media Option, and
want to serve content to it from AOLserver ... I've got a module that
"works" (but is far from complete) that does it. :-)
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
--
Vlad Seryakov
703 488-2173 office
[EMAIL PROTECTED]
http://www.crystalballinc.com/vlad/
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 06:55, Dossy Shiobara wrote: > > I could very well be wrong. If someone is interested in finding an > off-the-shelf, or implementing from scratch, a load generator for one of > these non-HTTP protocols (say, SMTP) ... I'd like to have a bake-off. > Implemented in pure Tcl, and implemented using one of the two proposed > patches. Then, benchmark the two and see if there's a measurable > difference. Alternatively, you could implement a web server using your tivo server method and use apache bench to test it against stock AOLserver. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 14:55, Dossy Shiobara wrote: > Certainly. My own input into the discussion shouldn't preclude others > from participating. However, I tend to believe that my opinion also > counts and deserves to be voiced along with everyone else's for > consideration. Absolutely. Everybody's (that is, anybody actively involved in AS project, I'd say) meaning counts. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 03:28, Bas Scheffers wrote: > Very interesting. After only reading your nutshell descriptions, to me > what would seem like a nice way of handling this is lightweight plumbing > in C and the bulk in Tcl. This is the essence of my proposed solution. You are required to write a parser in C, you can handle the processing in either C or Tcl in the normal fashion by registering a proc. Strictly speaking, you're not even required to parse the request in C. The minimum requirement is that you identify the end of the request (this would be a line end for an echo server request). You can do the actual parsing in Tcl within your registered proc; the content of the entire request is available by calling [ns_conn content]. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 05:46, Dossy Shiobara wrote: > > So far, this has been working really well for me. No modification to > the core involved, and it's implemented in very few LOC overall. Using > the same pattern, I could easily implement an SMTP, IMAP, etc., server > in pure Tcl. I'm not sure what the real performance difference there > would be compared to the two proposals that require C code changes in > the core: as long as the actual "functionality" is implemented in a Tcl > callback that gets invoked from the C code, the performance difference > is probably negligible. The performance difference would certainly be noticeable. In your example code a new thread is spawned for each connection. This can take a non trivial amount of time, especially if a new interp has to be initialized. Each of your connection threads and all it's resources are committed for the life of the connection. It will remain committed until the client disconnects. There are some functional differences too: no access logging; none of the niceties of registered procs or filters; no caching of returned files, etc. etc. None of which matters for a toy like a Tivo server. An SMTP or IMAP server implemented like this wouldn't be worth the bother. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 03:59, Andrew Piskorski wrote: > On Tue, Aug 17, 2004 at 10:32:56AM +0200, Zoran Vasiljevic wrote: > > > Vlad's patch implements entirely new socket-level driver and sticks > > the whole add-in functionality in the driver itself, effectively > > > Stephen's patch integrates into the AS by adding new API call > [...] > > from a connection socket. This way a new protocol connect handling > > looks pretty much like regular http. Hence it utilitzes all of the > > Are there any types of protocols that Vlad's approach can handle while > Stephen's cannot? E.g., protocols that look very different from HTTP? > If so, would it make sense to support BOTH methods in AOLserver? Although the example protocol driver for POP3 generates what looks like a (basic) HTTP request, there is no requirement to do so, it was just convenient. If you were handling some binary protocol, the minimum you would have to do in the parse proc would be identify the end of the message, and set the request line 'MYPROTO /'. You would then register a proc for that method and URL which calls Ns_ConnContent to get access to the payload of the message. The request line becomes a dummy, but it may still be useful. For example you can use it to force processing from one or another connection pool. There is the restriction at the moment that protocols which require pipelining (sending message 2 before receiving the answer to message 1), and by extension protocols which don't need an answer to all requests, aren't supported. I'd like to add that though, it's a requirement of HTTP/1.1 and a handy speed boost. > Also, this is all still only for protocols on top of TCP/IP, right? > No support for UDP or any other sort sockets? Vlad has submitted a patch which adds support for this to the binder: http://sourceforge.net/tracker/index.php?func=detail&aid=463625&group_id=3152&atid=353152 I think the most useful feature here would be unix domain sockets for something like a fastcgi module. This would be transparent to existing socket drivers. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: > The thing is: you may couple tclhttpd with some db-libraries > plus Tcl threading extension and get more-less a fully functional, > performant multi-threaded webserver in no-time. > Yet, we still use AS. The question is: why? Unless that was meant to be a rhetorical question, I'd like to hear your answer to the question. Why do I use AOLserver? Because, essentially, someone ELSE has coupled the Tcl interpreter with an HTTP server, a DB API, Tcl threading extensions, and has tested the whole combination to some degree -- so that I didn't have to. To do so myself, from scratch, would be a large investment in time and effort which would unnecessarily duplicate the efforts of others. With the speed-ups in Tcl itself through optimizations, the performance of pure Tcl solutions will gradually improve. The ease of Tcl development usually outweighs the performance gained by a C solution. I see two "classes" of people using AOLserver for implementing non-HTTP servers: 1) Low to medium performance requirements, short development timeline, need for rapid development, programmers with low to average technical expertise. 2) High performance requirements, highly skilled programmers, moderate development timeline, scalability a major concern. For class #1, a pure Tcl solution should be more than adequate for even the production implementation. For class #2, you're arguing that these people can justify the investment of writing and maintaining C code running in AOLserver? I'm arguing that with the advances in Tcl's performance today, it may be unnecessary to implement as many things in C today as it used to be a year or more ago. In a nutshell: I am guessing that any project whose performance requirements for a non-HTTP server are truly high enough to require a C-based solution will not work even if shoehorned into AOLserver, even with the two patches that have been proposed now. I could very well be wrong. If someone is interested in finding an off-the-shelf, or implementing from scratch, a load generator for one of these non-HTTP protocols (say, SMTP) ... I'd like to have a bake-off. Implemented in pure Tcl, and implemented using one of the two proposed patches. Then, benchmark the two and see if there's a measurable difference. > Let other people speak-up and we'll see the consensus. > Then, we may act accordingly. Certainly. My own input into the discussion shouldn't preclude others from participating. However, I tend to believe that my opinion also counts and deserves to be voiced along with everyone else's for consideration. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 13:46, Dossy Shiobara wrote: > Just the other night, I had the need to implement a non-HTTP server in > AOLserver in order to implement the TiVo "beacon" connection for the > Home Media Option (MHO) feature of the Series2 TiVo. I thought about > both Vlad's and Stephen's approaches, and decided I'd rather stay > pure-Tcl instead. Here's what I came up with: Well, of course... If you only have a hammer than everyting looks like a nail! I do not recall who coined this phrase in the first place, but it can be applied very good here. The non-http functionality of the AS can be implemented with Lego-pieces already there, no doubt about it. Works fine for prototyping. Not more. The thing is: you may couple tclhttpd with some db-libraries plus Tcl threading extension and get more-less a fully functional, performant multi-threaded webserver in no-time. Yet, we still use AS. The question is: why? Let other people speak-up and we'll see the consensus. Then, we may act accordingly. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On 2004.08.17, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote:
>
> So, all of you interested in extending AS to handle non-http
> protocols, please go, have a look and give some feedback.
Just the other night, I had the need to implement a non-HTTP server in
AOLserver in order to implement the TiVo "beacon" connection for the
Home Media Option (MHO) feature of the Series2 TiVo. I thought about
both Vlad's and Stephen's approaches, and decided I'd rather stay
pure-Tcl instead. Here's what I came up with:
proc tivo.init {} {
# ...some setting of nsv config vars here...
ns_thread begindetached tivo.beaconThread
}
proc tivo.beaconThread {} {
set address [nsv_get tivo address]
set beaconport [nsv_get tivo beaconport]
ns_log notice "TiVo beacon listening on $address:$beaconport"
set master [ns_socklisten $address $beaconport]
while {![nsv_get tivo shutdown]} {
foreach {rfd wfd} [ns_sockaccept $master] break
ns_chan create $rfd $rfd
ns_chan create $wfd $wfd
ns_thread begindetached [list tivo.beacon $rfd $wfd]
}
close $master
}
tivo.beacon {rfd wfd} {
ns_chan get $rfd
ns_chan get $wfd
# ... do actual beacon exchange ...
# connection must remain open, so we just poll until the remote
# side disconnects.
while {![nsv_get tivo shutdown]} {
set sel [ns_sockselect -timeout 2 $rfd {} {}]
if {[llength [lindex $sel 0]]} {
read $rfd 1
if {[fblocked $rfd] || [eof $rfd]} {
ns_log notice "TiVo: Beacon disconnected."
break
}
}
}
close $rfd
close $wfd
}
The gist of this is that:
1. Can't use [ns_socklistencallback] since our connection handler keeps
the connection open, and that would block the thread that executes
the callback handler.
2. At server start-up, we create a thread to just accept connections
which creates new threads to process those connections.
3. We use [ns_chan] to detach/attach Tcl channels to the socket fd's in
order to pass them from the master thread to the slave threads.
So far, this has been working really well for me. No modification to
the core involved, and it's implemented in very few LOC overall. Using
the same pattern, I could easily implement an SMTP, IMAP, etc., server
in pure Tcl. I'm not sure what the real performance difference there
would be compared to the two proposals that require C code changes in
the core: as long as the actual "functionality" is implemented in a Tcl
callback that gets invoked from the C code, the performance difference
is probably negligible.
On a side note: if you have a Series2 TiVo with Home Media Option, and
want to serve content to it from AOLserver ... I've got a module that
"works" (but is far from complete) that does it. :-)
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 11:28, Bas Scheffers wrote:
> Very interesting. After only reading your nutshell descriptions, to me
> what would seem like a nice way of handling this is lightweight plumbing
> in C and the bulk in Tcl. Example:
>
> Create a socket driver that, on connection to it's IP/port, gets an
> interpreter thread and in it runs the procedure configured for it, ie:
>
> proc smtp {in out from_addr args} {smtp code}
>
> When the proc exits, the connection closes and, just like HTTP, the
> interpreter is cleaned up a bit and put back in the pool.
>
> This would make it easy to implement any protocol you want, possibly using
> expect. It could make nsd a very interesting core for building any
> client/server apps.
>
This is basically what Vlad's patch is doing.
Please go ahead and checkout both proposals
to see real pros/cons of each of them.
Althought Vlad's patch is much simpler I somehow think
that Stephens one is cleaner and more powerful on the
long run.
Zoran
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tuesday 17 August 2004 11:59, Andrew Piskorski wrote: > On Tue, Aug 17, 2004 at 10:32:56AM +0200, Zoran Vasiljevic wrote: > > > Vlad's patch implements entirely new socket-level driver and sticks > > the whole add-in functionality in the driver itself, effectively > > > Stephen's patch integrates into the AS by adding new API call > [...] > > from a connection socket. This way a new protocol connect handling > > looks pretty much like regular http. Hence it utilitzes all of the > > Are there any types of protocols that Vlad's approach can handle while > Stephen's cannot? E.g., protocols that look very different from HTTP? > If so, would it make sense to support BOTH methods in AOLserver? Both approaches can handle arbitrary TCP protocols. > > Also, this is all still only for protocols on top of TCP/IP, right? > No support for UDP or any other sort sockets? Stepens patch sits atop of existing nssock driver hence it is doing any protocols nssock can (TCP only for now). But there is nothing which forbids you to make a UDP-level socket driver and integrate it in the AS and then use Stephens patch to add your own upper-level processing. Vlad's patch does replace the nssock entirely, leaving you to implement whatever you want in the driver. It has very limited interaction to AS machinery really. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, Aug 17, 2004 at 10:32:56AM +0200, Zoran Vasiljevic wrote: > Vlad's patch implements entirely new socket-level driver and sticks > the whole add-in functionality in the driver itself, effectively > Stephen's patch integrates into the AS by adding new API call [...] > from a connection socket. This way a new protocol connect handling > looks pretty much like regular http. Hence it utilitzes all of the Are there any types of protocols that Vlad's approach can handle while Stephen's cannot? E.g., protocols that look very different from HTTP? If so, would it make sense to support BOTH methods in AOLserver? Also, this is all still only for protocols on top of TCP/IP, right? No support for UDP or any other sort sockets? -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
Very interesting. After only reading your nutshell descriptions, to me
what would seem like a nice way of handling this is lightweight plumbing
in C and the bulk in Tcl. Example:
Create a socket driver that, on connection to it's IP/port, gets an
interpreter thread and in it runs the procedure configured for it, ie:
proc smtp {in out from_addr args} {smtp code}
When the proc exits, the connection closes and, just like HTTP, the
interpreter is cleaned up a bit and put back in the pool.
This would make it easy to implement any protocol you want, possibly using
expect. It could make nsd a very interesting core for building any
client/server apps.
Just my $0.02!
Bas.
Zoran Vasiljevic said:
> Hi!
>
> I would like to start a discussion which will eventually lead
> to one of the two proposals listed below (one from Vlad Seryakov and the
other from Stephen Deasey) included in the core-server
> distribution.
>
> Both proposals look promising and both allow us to add support
> for arbitrary protocols (pop, imap, ftp...) to the AS, augmeting the
existing (built-in) http protocol.
>
> Vlad Seryakov's patch can be found on:
> http://sourceforge.net/tracker/index.php?func=detail&aid=726288&group_id=3152&atid=353152
>
> In a nutshell:
> Vlad's patch implements entirely new socket-level driver and sticks the
whole add-in functionality in the driver itself, effectively bypassing
the AS socket and/or ssl driver. It is minimaly invasive in terms of
core-server changes as it puts all of the new protocol parsing logic
into the driver itself.
>
>
> Stephen Deasey's patch can be found on:
> http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152
>
> In a nutshell:
> Stephen's patch integrates into the AS by adding new API call
> Ns_RegisterParseProc() which allows module authors to register a proc
which will be called by the driver thread to parse the data from a
connection socket. This way a new protocol connect handling looks pretty
much like regular http. Hence it utilitzes all of the recent
optimizations in the core (read-aead threads, async-reads) and runs
atop existing sock and ssl drivers.
>
>
> My personal opinion is that Stephen's patch is more versatile in the way
it integrates with other AS components. It brings more
> options to the protocol developer. It is relatively simple, not
> too invasive and I would vote we integrate this patch into the
> 4.1+ dvelopment branch.
>
> So, all of you interested in extending AS to handle non-http
> protocols, please go, have a look and give some feedback.
>
> Thanks,
> Zoran
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.
>
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
[AOLSERVER] Support for non-HTTP protocols
Hi! I would like to start a discussion which will eventually lead to one of the two proposals listed below (one from Vlad Seryakov and the other from Stephen Deasey) included in the core-server distribution. Both proposals look promising and both allow us to add support for arbitrary protocols (pop, imap, ftp...) to the AS, augmeting the existing (built-in) http protocol. Vlad Seryakov's patch can be found on: http://sourceforge.net/tracker/index.php?func=detail&aid=726288&group_id=3152&atid=353152 In a nutshell: Vlad's patch implements entirely new socket-level driver and sticks the whole add-in functionality in the driver itself, effectively bypassing the AS socket and/or ssl driver. It is minimaly invasive in terms of core-server changes as it puts all of the new protocol parsing logic into the driver itself. Stephen Deasey's patch can be found on: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 In a nutshell: Stephen's patch integrates into the AS by adding new API call Ns_RegisterParseProc() which allows module authors to register a proc which will be called by the driver thread to parse the data from a connection socket. This way a new protocol connect handling looks pretty much like regular http. Hence it utilitzes all of the recent optimizations in the core (read-aead threads, async-reads) and runs atop existing sock and ssl drivers. My personal opinion is that Stephen's patch is more versatile in the way it integrates with other AS components. It brings more options to the protocol developer. It is relatively simple, not too invasive and I would vote we integrate this patch into the 4.1+ dvelopment branch. So, all of you interested in extending AS to handle non-http protocols, please go, have a look and give some feedback. Thanks, Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
